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ABSTRACT 


Security  of  a  Forward  Operating  Base  (FOB)  is  of  high  interest  and  operational 
importance  to  the  U.S.  military  and  allied  forces.  The  Situational  Awareness  for 
Surveillance  and  Interdiction  Operations  (SASIO)  model  simulates  the  operational 
tasking  of  a  single  Unmanned  Aerial  Vehicle  (UAV)  and  a  ground-based  interceptor  that 
are  designed  to  search,  identify,  and  intercept  potential  hostile  targets  prior  to  reaching 
the  FOB.  This  thesis  explains  the  SASIO  model  and  its  implementation  in  JAVA.  This 
theoretical  model  leverages  Design  of  Experiments  (DOE),  which  varies  multiple 
characteristics  of  the  system  to  explore  insights  for  the  tactical  employment  of  UAV  and 
interceptor  to  combat  potential  hostile  actions  against  a  predefined  area  of  interest. 
Designed  screening  simulation  experiments  identifies  influential  factors  to  provide 
guidance  for  tactical  employment  of  Blue  Eorce  assets,  as  well  as  provide  alternative 
means  to  influence  Red  force  behavior  in  a  beneficial  manner.  This  thesis  analyzed  the 
effects  of  the  influential  factors  with  respect  to  the  percentage  of  threats  interdicted,  time 
to  acquire  threats,  and  mean  distance  away  from  the  EOB  that  the  threats  were 
interdicted.  Through  analytical  techniques,  a  quantifiable  measure  of  the  employment 
strategy  for  the  UAV  and  ground-based  interceptor  was  achieved. 
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EXECUTIVE  SUMMARY 


Security  of  a  Forward  Operating  Base  (FOB)  is  of  high  interest  and  operational 
importance  to  the  U.S.  military  and  allied  forces.  The  Situational  Awareness  for 
Surveillance  and  Interdiction  Operations  (SASIO)  model  simulates  the  operational 
tasking  of  a  single  Unmanned  Aerial  Vehicle  (UAV)  and  a  ground-based  interceptor  that 
is  designed  to  search,  identify,  and  intercept  potential  hostile  targets  prior  to  reaching  the 
FOB.  This  thesis  explains  the  SASIO  model  and  its  implementation  in  JAVA.  The  goal 
of  this  thesis  is  to  provide  insight  into  influential  factors  that  lead  to  developing  the 
matrix  pertaining  to  the  percentage  of  threats  interdicted,  time  to  acquire  the  threats  and 
distance  away  from  the  FOB  that  the  threats  were  interdicted. 

Insights  and  Recommendations 

Utilizing  analytic  techniques,  the  thesis  explored  the  employment  of  Blue  Force 
assets,  as  well  as  Blue  Force  action  that  is  external  to  their  asset  employment  strategies. 
The  sensor  characteristics  of  the  UAV  minimally  impacted  the  response  variables  studied 
for  this  scenario.  However,  the  UAV  as  a  surveillance  asset  did  provide  the  ground-based 
interceptors  an  increased  detection  range  allowing  for  a  marginal  increase  in  interdicting 
the  threats. 

Utilizing  a  single  UAV  and  ground-based  interceptor,  the  actions  of  whether  the 
ground-based  interceptor  patrolled  or  remained  stationary  at  the  FOB  dramatically 
influenced  the  percentage  of  threats  interdicted,  as  well  as  the  mean  distance  away  from 
the  FOB.  If  the  ground-based  interceptor  patrolled,  the  percentage  of  threats  interdicted 
decreased  by  53%,  while  the  mean  distance  away  from  the  FOB  for  interdiction  increased 
by  11%.  Therefore,  through  analytical  techniques,  a  quantifiable  measure  of  the 
employment  strategy  for  the  ground-based  interceptor  was  achieved. 

The  most  influential  factor  with  respect  to  all  of  the  responses  is  the  enemy’s 
speed  as  they  traverse  towards  the  FOB.  As  the  speed  of  the  enemy  increased,  each 


response  degraded.  Therefore,  if  external  eheekpoints  are  provide  by  indigenous 
personnel  to  reduee  the  enemy’s  traversal  speed,  then  the  Blue  Foree  ean  more  effeetively 
interdiet  the  threat. 

Scenario  and  Concepts  of  Operations 

The  probability  and  stoehastie  models  diseussed  in  this  thesis  inelude  multiple 
moving  neutral  and  threat  ground  objeets  and  a  surveillanee  and  interdietion  foree 
eomprising  a  single  UAV  for  surveillanee  and  identifieation  and  a  single  ground-based 
intereeptor  designed  to  intereept  and  elear  any  hostile  objeets.  Upon  initialization  of  the 
seenario,  the  threats  and  neutral  objeets  arrive  in  the  area  of  interest  randomly  and 
traverse  towards  the  FOB.  The  goal  of  the  Blue  Foree  is  to  deteet  and  identify  the  threats, 
issue  eommands  to  the  ground  interdietor,  and  elear  the  threats  prior  to  reaehing  the  FOB. 
Onee  the  UAV  deteets  and  identifies  (with  some  identifieation  error)  the  threats,  then  the 
intereeptor’ s  responsibility  is  to  traverse  towards  the  threat  in  order  to  elear  that  threat. 
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I.  INTRODUCTION 


A,  PROBLEM/  NAVY  INTEREST 

Surveillance  and  Interdiction  (SI)  problems  are  of  great  interest  to  the  United 
States  military  due  to  the  necessity  to  obtain  pertinent  information  of  potential  hostile 
threats  and  to  neutralize  those  threats  upon  their  correct  identification.  The  military 
utilizes  numerous  assets  such  as,  but  not  limited  to:  Unmanned  Aerial  Vehicles  (UAVs), 
Ground  Based  Operational  Surveillance  Systems  (GBOSS),  and  satellite  imagery  to  aid 
in  categorizing  the  entities’  actions  as  threats  or  not.  Upon  detection  and  identification  of 
possible  threats,  the  military  must  take  action  to  minimize  and/or  perturb  the 
aforementioned  threats  to  maintain  security  within  the  regions  of  operation.  Linking  the 
surveillance  and  interdiction  efforts  together  is  an  operational  hurdle  that  must  be 
overcome  to  ensure  the  safety  of  our  military  forces. 

The  attack  on  the  USS  Cole  (United  States  Department  of  Defense,  2001)  may  be 
classified  as  an  SI  problem,  and  it  illustrates  the  necessity  to  process  relevant  information 
in  a  rapid  manner.  Given  the  atmosphere  surrounding  this  event,  proper  patrolling  of  the 
surrounding  waters,  as  well  as  inspection  of  any  approaching  vessels  prior  to  allowing  the 
vessel  to  tie  up  to  the  ship,  may  have  thwarted  this  attack.  Another  attack  that  falls  within 
this  class  of  problems  is  the  Suicide  Vehicle-borne  improvised  explosive  device  (VBIED) 
attacks  on  an  Iraqi  Army  checkpoint  (Multi-National  Corps — Iraq,  2008).  Quick  Reaction 
Force  (QRF)  units  and  checkpoints  were  established  to  prevent  such  an  attack;  however, 
the  actions  by  embassy  personnel  were  not  able  to  properly  identify  and  detect  the 
VBIFD.  The  overall  problem,  as  noted  above,  is  the  fact  that  although  information  and 
data  surrounding  an  operational  environment  may  be  available,  the  ability  to  take  action 
to  mitigate  such  attacks  requires  relatively  quick  analysis  that  produces  feasible  and 
desired  results.  In  this  context,  for  example,  “the  true  operational  art  behind  that  strike 
[airstrike  that  killed  Zarqawi]  was  a  multidisciplined  intelligence,  surveillance,  and 
reconnaissance  endeavor  with  agile  special  operating  forces  ...  resulting  in  a  find-fix- 
finish  operation”  (Cantrell  et  al.  2008). 
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One  such  instance  of  the  necessity  for  find-fix-finish  operations  is  for  the 
installation’s  protection.  Suppose  the  military  is  operating  within  the  confines  of  a 
Forward  Operating  Base  (FOB)  within  a  region.  The  security  of  this  FOB  is  paramount 
due  to  the  military  operations  that  take  place  in  the  region.  Military  scouts,  such  as 
UAVs,  may  be  flown  for  surveillance  purposes  to  allow  the  FOB  to  extend  its  detection 
and  identification  range  to  ensure  an  adequate  standoff  distance  from  any  threats  that  may 
try  to  attack  and/or  infiltrate  the  base.  The  data  obtained  from  the  UAV  can  either  be 
automated  via  tracking  and  identification  algorithms,  which  allows  the  UAV  to  perform 
its  operations  without  the  need  of  a  human  to  make  complex  decisions,  or  employ  the 
current  “human-in-the-loop”  process,  which  requires  a  human  to  process  mass  data  in 
either  live-feed  video  or  still  imagery.  Based  on  the  data,  the  commander  must  make 
complex  decisions  to  counteract  actions  that  may  be  threatening  the  installation. 

The  decision  to  engage  an  enemy  must  be  made  based  on  quick-tum  analysis  of 
surveillance  data  such  that  the  entity  is  ideally  cleared  outside  of  a  predefined  standoff 
distance  away  from  the  installation  to  ensure  minimal  loss  to  life  and  property.  Current 
doctrine  requires  action  on  any  potential  threat;  this  ensures  that  all  entities  are  cleared 
through  interdiction.  However,  following  this  doctrine,  manpower  is  not  utilized  in  an 
efficient  manner,  and  if  limited  resources  exist,  then  clearing  all  entities  may  not  be 
possible.  This  thesis  illustrates  how  the  SASIO  simulation  model  can  be  used  as  an 
analysis  tool  to  provide  commanders  with  insight  on  employment  techniques  to  counter 
potential  threats  given  limited  surveillance  and  interdiction  resources  to  protect  a  FOB  (or 
any  other  location  that  requires  protection). 

B,  BENEFITS  OF  STUDY 

Surveillance  and  Interdiction  problems  appear  in  many  forms.  The  military  is 
relatively  good  at  gathering  the  data  from  surveillance  assets,  but  linking  potential  flags 
from  one  data  set  to  another  to  obtain  a  course  of  action  during  military  operations  is 
relatively  slow.  During  military  operations.  Combatant  Commanders  are  either  not  given 
concise  information,  which  may  or  may  not  require  immediate  action,  or  they  are 
presented  with  data  requiring  actions  that  ultimately  exhaust  the  assets  without 
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investigating  all  of  the  potential  threats  or  options.  Understanding  and  aeting  on  only 
pertinent  intelligenee  in  a  timely  manner  ean  alleviate  exhaustion  of  assets,  as  well  as 
minimize  the  risk  assoeiated  with  performing  extraneous  interception  missions  for  low- 
threat  targets.  This  thesis  explains  the  Situational  Awareness  for  Surveillance  and 
Interdiction  Operations  (SASIO)  model  and  how  Combatant  Commanders  can  use  this 
model  in  support  of  relevant  SI  settings.  Situational  awareness,  as  defined  for  this  thesis, 
represents  the  knowledge  and  understanding  of  the  environment  and  the  actors  within  that 
environment.  Use  of  the  SASIO  model  supports  either  real-time  employment  strategies 
(decision  support)  or  robust  design  strategies  (analysis  tool)  to  maximize  the  employment 
of  surveillance  and  interceptor  assets.  The  pertinent  intelligence  gathered  from  the 
employment  of  such  assets  could  enable  more  effective  interdiction  of  threats  as  they 
may  arise  or  before  they  engage. 

The  SASIO  model  was  developed  to  provide  a  decision  support  tool  based  on 
currently  available  and  forward-deployed  assets.  SASIO  may  utilize  current 
characteristics  of  surveillance  and  QRF  assets  to  provide  an  employment  strategy  via 
multiple  simulation  iterations,  as  well  as  provide  real-time  support  through  simulation 
iterations  based  on  the  ever-changing  theater.  The  SASIO  model  can  support  many 
classes  of  SI  contexts  ranging  from  the  aforementioned  FOB  protection,  border 
protection,  counter  drug  operations  and  Maritime  operations  with  regards  to  anti-pirating 
operations.  These  types  of  problems  share  common  components  that  comprise  any  SI 
missions: 

•  Utilize  surveillance  assets  to  patrol  the  area  to  identify  threats  pertaining 
to  the  security  of  the  area  of  operation 

•  Detect  and  classify  threats  and  neutral  actors  based  on  the  intelligence 
gathered  from  the  surveillance  asset 

•  Transmit  the  information  obtained  by  the  surveillance  asset  to  interdiction 
assets 

•  Interdiction  assets  can  either  investigate  and/or  interdict  the  actor  based  on 
the  transmitted  report  or  wait  for  additional  reports 

Utilizing  this  framework,  while  combining  the  effects  of  probabilistic  sensing  and 
motion  by  Blue  Force  assets,  the  evolution  of  this  model  can  provide  Commanders  with  a 
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decision  support  tool.  SASIO  links  the  gathering  of  information  to  the  execution  of 
actions  that  must  be  performed  to  ensure  security  of  an  area  through  recommendations 
and  potential  outcomes  of  a  failed  action. 

In  support  of  such  a  tool,  SASIO  provides  insight  into  the  factors  of  relevance  for 
performing  said  tasks,  such  as:  speed  of  each  asset  (UAV,  QRF),  sensor  characteristics 
(false  positive  and  missed  detection)  rates,  when  the  QRF  takes  action  and  also  aid  in 
determining  a  secure  location  for  a  FOB  within  a  hostile  region.  SASIO  may  be  iterated 
through  with  varying  factors  relatively  quickly,  which  could  allow  an  analyst  to  provide 
asset  employment  recommendations  based  on  the  current  operational  environment  being 
studied.  Combining  this  fact  and  performing  sensitivity  analysis  on  these  factors,  the 
potential  shortcomings  of  current  technology  and  doctrine  (e.g..  Tactics,  Techniques  and 
Procedures)  may  be  evaluated  to  provide  meaningful  insight  in  the  context  of  SI 
problems,  and  revised  as  necessary. 

This  thesis  makes  use  of  the  SASIO  model  to  explore  the  effectiveness  of  a  single 
UAV  and  a  single  QRF  with  regards  to  the  protection  of  a  FOB.  Numerous  factors  are 
examined  in  order  to  analyze  their  significance  in  the  proper  employment  of  these  forces 
to  combat  a  potential  threat  to  the  FOB.  This  threat  is  characterized  as  a  vehicle-home 
improvised  explosive  devise  that  can  damage  the  FOB  even  if  the  threat  does  not  reach 
the  FOB.  As  the  distance  from  the  FOB  to  the  interdicted  threat  is  increased,  then  the 
destmctive  capability  of  the  threat  is  decreased.  The  scenario  makes  use  of  the 
topographical  constraints  and  road  network  found  at  Camp  Roberts  Army  National  Guard 
installation  in  California. 

This  work  is  motivated  by  ongoing  field  experimentation  efforts  via  the  Naval 
Postgraduate  School  and  the  United  States  Special  Operations  Command  (USSOCOM) 
Field  Experimentation  Cooperative  program  (Bordetsky  et  ah,  2009).  The  current 
experimental  scenario  includes  a  forward-deployed,  12-man  team  designed  to  protect 
their  command  installation  (Special  Operations  team  house)  while  performing 
intelligence,  surveillance  and  reconnaissance  (ISR)  operations  via  UAV  assets.  Through 
use  of  limited  resources,  primarily  minimal  surveillance  assets,  employment  tactics  for 

the  UAV  based  on  varying  theater  configurations  (SOF  team  house  placements)  are 
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evaluated.  Based  on  time  eonstraints  (e.g.,  long  lead  time  required  to  perform  a  single 
live  experiment),  the  SASIO  model  ean  be  used  in  eonjunetion  with  eurrent  field 
experiments  to  provide  analysis  based  on  eurrent  taeties  utilized,  as  well  as  provide 
insight  into  alternative  employment  strategies  to  maximize  the  proteetion  of  an 
installation. 

The  SASIO  simulation  engine  ineorporates  a  modular  type  framework.  The  user 
may  seleet  any  eurrently  built-in  motion  or  interdietion  models,  or  may  ereate  their  own 
to  test  out  new  employment  teehniques  for  both  the  surveillanee  and  interdietion  assets 
alike.  This  feature  is  useful  given  that  an  employment  for  one  theater  may  not  be  effieient 
or  appropriate  for  another.  This  modularity  ensures  that  the  user  is  not  limited  to  a  single 
teehnique,  but  is  able  to  evaluate  a  multitude  of  strategies  to  eombat  an  evolving  enemy. 

C.  RELATED  WORK 

There  have  been  reeent  researeh  efforts  with  respeet  to  the  employment  of  UAVs 
for  ISR  type  missions.  Prior  to  performing  ISR  tasks,  the  underlying  elass  of  problem 
must  be  understood.  Mission  planning  and  employment  must  take  into  aeeount  aspeets  of 
elassieal  seareh  modeling  as  it  applies  to  the  imperfeet  sensor  eharaoteristios  of  the  UAV. 
Washburn  (2002)  studied  the  propagation  of  errors  in  a  simulation  due  to  an  imperfeet 
sensor.  Berner  (2004)  studied  sensor  oharaeteristie  of  varying  UAVs  to  obtain  situational 
awareness  for  proteetion  of  a  high  value  target  while  Raffetto  (2004)  foeused  primarily 
on  the  faetors  of  airspeed  and  loiter  times  for  the  UAV.  Berner’s  (2004)  study  analyzed 
how  a  broad  area  seareh  UAV  with  imperfeet  sensor  eharaoteristios  may  be  used  in 
eonjunetion  with  a  better  deteotion  and  identifioation  UAV  with  a  relatively  lower 
deteotion  range  to  inorease  the  ooverage  surrounding  a  partioular  mission  region. 

Onoe  situational  awareness  is  aohieved  for  a  region,  the  ability  to  allooate  a  QRF 
and  proper  employment  of  alternate  surveillanee  assets  must  be  performed  to  aoquire 
targets  within  a  short  time  after  identifioation.  Chung  et  al.  (2009)  formulated  a  Mixed- 
Integer  program  utilizing  Bayesian  updates  to  perform  the  task  of  allooating  surveillanee 
and  QRF  assets  to  rapidly  identify  potential  targets.  Similarly,  this  thesis  foouses  on  the 
taotioal  uses  for  eaoh  asset.  Other  methods  to  rapidly  interdiot  potential  targets  utilize  a 
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Subject  Matter  Expert  (SME)  to  manually  interject  proposed  enemy  trajectories  based  on 
experience  in  the  region.  Kress  et  al.  (2007)  demonstrated  significant  improvements  in 
search  performance  through  the  implementation  of  a  “two-stage  stochastic  Integer  Einear 
program”  in  conjunction  with  SME  inputs.  Although  this  is  an  interesting  improvement 
to  the  SI  class  of  problems  (consisting  of  an  increase  in  50%  detection  opportunities 
based  on  a  preplanned  generated  UAV  routing  provided  by  the  SME),  the  paper  does  not 
address  variations  in  factors  as  they  pertain  to  surveillance,  such  as:  sensor 
characteristics,  sensor  footprint,  sensor  loiter  time  and  platform  speed.  SASIO  seeks  to 
investigate  such  factors,  as  well  as  others,  to  obtain  a  more  robust  design  for  coordinated 
employment  of  each  type  of  agent. 

Much  of  the  past  research  has  focused  on  the  surveillance  abilities  of  the  UAVs. 
Upon  a  UAV  observation,  the  UAV  transmit  the  location  of  the  threat  to  the  QRE  with 
some  time  delay  due  to  transmittal  distance.  With  this  time  delay,  an  error  in  the  exact 
location  of  the  threat  arises  and  the  location  must  be  offset  to  account  for  this  error  to 
provide  a  successful  interdiction.  Bertuccelli  et  al.  (2006)  illustrated  the  importance  of 
taking  into  account  variability  in  information  due  to  time  delays  for  UAV  search  and 
ground  interceptions.  The  author  used  Uncertain  Probability  Maps  (UPMs),  which 
accounted  for  poor  knowledge  and  propagations  of  the  target  location  throughout  the 
environment.  Through  recursive  calls  of  a  distribution  of  the  targets’  trajectories,  the 
maps  are  generated  and  updated  for  efficient  search.  In  this  thesis,  UAV  routes  are 
heuristics  based  on  a  probability  map  representation  of  situational  awareness  that  is 
updated  with  real  time  observations. 

Eor  the  purposes  of  this  thesis,  SASIO  performs  the  tasks  of  SI  based  on  a  single 
UAV  for  surveillance  and  a  single  QRE.  Other  past  works  have  studied  the  case  where 
multiple  assets  were  utilized.  Yan  et  al.  showed  UAV  performance  of  multiple  tasks 
based  on  their  capabilities  such  as:  searching,  confirming,  attacking,  and  performing 
Battle  Damage  Assessment  (BDA)  (Yan  et  ah,  2003).  The  authors  used  deterministic 
Discrete  Event  Simulation  (DES),  continuing  efforts  until  all  targets  were  eradicated. 
This  model  illustrated  that  there  exists  a  point  in  which  an  increase  in  the  number  of 
UAVs  showed  diminishing  returns.  Also,  an  active  search  algorithm  for  the  UAVs  was 
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more  effective  than  a  random  search  in  eradication  of  targets.  In  contrast,  the  SASIO 
project  utilizes  a  time-stepped  simulation,  integrate  imperfect  observations  with  Bayesian 
updates,  and  vary  numerous  factors  pertaining  to  Red  and  Blue  Force  assets  and  theater. 

D,  THESIS  ORGANIZATION 

The  main  contributions  of  this  thesis  are  the  development  of  the  SASIO  model  as 
well  as  the  analysis  and  resulting  insights  for  tactical  employment  of  Blue  Force  assets  in 
an  SI  context.  Chapter  II  outlines  the  basic  elements  used  in  constructing  the  SASIO 
model.  In  Chapter  III,  the  thesis  delves  into  the  details  of  the  model  while  highlighting 
potential  variations  in  the  model  to  explore  alternate  scenarios.  Chapter  IV  discusses  the 
scenario  analysis  and  presents  the  reader  with  the  results  of  the  analysis,  as  well  as  the 
operational  and  tactical  insights  that  may  be  obtained  from  the  analysis.  Chapter  V 
provides  a  summary  of  this  research  and  the  conclusions  as  well  as  present  avenues  for 
future  research. 

E.  SCENARIO 

Within  the  area  of  interest  (AOI),  the  military  has  created  a  FOB  where  military 
operations  for  the  immediate  region  based.  There  are  three  roads  that  we  consider  three 
entities  entering  this  area,  one  of  which  is  a  threat,  while  the  other  two  are  neutral.  Each 
entity  enters  the  AOI  and  traverses  the  road  toward  the  FOB  at  a  constant  speed. 

The  assets  in  place  to  protect  the  FOB  (Blue  Force)  consist  of  a  single  UAV  for 
surveillance  and  a  single  QRF  to  interdict  and  clear  the  threat.  The  UAV  performs 
surveillance  along  fixed  radii  paths  from  the  FOB.  The  purpose  of  the  UAV  is  to  give  the 
Blue  Force  the  ability  to  identify  a  threat  while  at  an  increased  standoff  distance  from  the 
FOB.  Upon  a  detection  and  identification,  the  UAV  issues  a  report  for  the  QRF  to 
interdict  the  threat. 

The  QRF  performs  a  patrol  surrounding  the  FOB  at  a  distance  of  250  m  along  the 
road  network.  The  QRF  has  the  ability  to  perform  surveillance,  as  the  patrol  is  occurring 
and  interdict  threats  that  are  within  this  surveillance  range.  Therefore,  the  QRF  provides  a 
second  line  of  defense  to  the  FOB. 
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F.  ASSUMPTIONS 
Theater 

Road  Network  -  The  road  network  is  a  direeted  graph  from  entry  nodes  to  the 
AOI  towards  the  FOB. 

Node  Separation  -  Eaeh  node  is  250  m  apart. 

Theater  Map  -  Eaeh  node  has  a  probability  of  a  threat  in  that  node  and  eontinues 
to  be  updated  as  observations  are  made  by  the  Blue  Eoree  assets  via  a  Bayesian  updating 
proeess. 

Objects 

Expected  Arrival  Time  -  The  objeets  have  a  minimum  inter-arrival  time  of  one 

unit. 

Traversal  Speed  -  Threats  and  neutrals  traverse  at  a  fixed  speed  (unknown  to  the 
Blue  Eoree). 

Traversal  Direction  -  All  objeets  traverse  towards  the  EOB. 

Agents  (UAV) 

Surveillance  Path  -  The  UAV  flies  a  fixed  waypoint  paths  at  varying  distanees 
from  the  EOB. 

Detection/Identification  -  The  UAV  has  perfeet  detection  and  imperfect 
identification. 

Report  Issue  -  The  UAV  issues  a  report  upon  (possibly  erroneous)  positive  threat 
identification. 

Agents  (QRF) 

Patrol  -  The  QRE  performs  a  patrol  of  either  a  single  node  waypoint  path  away 
from  the  EOB  or  remains  stationary  at  the  EOB. 

Clearing  -  Upon  interdiction  of  a  threat,  the  QRE  detains  the  threat  and  clear  the 
threat  after  the  appropriate  time  has  elapsed. 

Visible  Cells  -  The  Blue  interdiction  force  has  the  ability  to  perform  surveillance 
within  its  current  cell  as  well  as  adjacent  cells  based  on  the  users  input. 

Detection/Identification  -  The  QRE  has  perfect  detection  and  identification 
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II.  MODEL  FORMULATION 


A,  OVERVIEW 

This  chapter  describes  the  development  of  the  theater,  scenario  and  taskforce.  The 
theater  governs  the  region  in  which  the  entities  interact.  For  this  thesis,  the  theater  is 
specified  as  a  region  patterned  after  Camp  Roberts  Army  National  Guard  facility  in 
California.  The  scenario  is  synonymous  with  the  objects  not  characterized  as  Blue  Force. 
These  objects  may  be  categorized  in  the  form  of  neutral  or  threat  objects.  The  last  portion 
of  the  model  is  the  taskforce  entities,  which  includes  all  Blue  Force  assets,  surveillance 
and  QRF  alike. 

B,  DESCRIPTION  OF  THEATER 

The  AOI  for  this  scenario  is  at  Camp  Roberts,  and  was  chosen  due  to  the  ability  to 
utilize  aerial  and  ground  based  assets  to  calibrate  the  model.  The  chosen  roads  form  a 
“Y”  shaped  network  with  the  FOB  located  where  the  three  roads  intersect.  Each  road  is 
discretized  into  nodes  separated  250  meters  apart  from  each  other  and  the  labeling  of  the 
nodes  is  arbitrary,  though  once  labeled,  each  node  is  maintained  and  known  to  all  Blue 
Force  entities  within  the  simulation  representing  a  common  operational  picture  (COP). 
The  scenario  theater  is  depicted  in  Figure  1. 
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Figure  1;  Camp  Roberts,  CA:  FOB  located  in  center  and  depicted  with  blue  diamond. 
Entry  nodes  are  illustrated  as  red  circles,  while  intermediate  nodes  are  in  white.  Each 
node  is  separated  in  250-meter  increments. 


The  graph  representing  the  road  network  is  an  undirected  graph  linking  each  node 
to  the  adjacent  nodes.  This  allows  for  objects.  Red  and  Blue  ground  elements  alike,  to 
transit  throughout  the  area  by  moving  from  a  single  node  to  its  adjacent  node.  The 
southern  road  has  only  1.5  km  in  total  distance  from  entrance  to  the  protected  EOB 
location,  while  the  other  northern  routes  are  3  km  in  total  distance.  This  was  determined 
based  on  airspace  constraints  within  Camp  Roberts.  The  separation  of  250  m  from  one 
node  to  the  next  was  chosen  due  to  the  nominal  sensor  footprint  of  the  aerial  asset 
utilized.  In  other  words,  the  asset,  while  flying  at  its  nominal  altitude,  provides  a  250  m 
wide  footprint,  thus  allowing  thorough  observation  of  a  single  node  during  each  pass. 
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C.  DESCRIPTION  OF  SCENARIO 

1 .  Arrival  Model  of  Vehicles 

Both  neutral  and  target  type  vehicles  enter  the  road  network  via  the  entry  nodes 
and  travel  towards  the  FOB.  Assume  that  any  object  traversing  the  network  away  from 
the  FOB  pose  no  threat  to  the  installation  and  can  be  disregarded.  Each  object’s  arrival  is 
assumed  independent  and  the  rate  associated  with  their  inter-arrival  times  to  begin  their 
transit  down  a  road  is  denoted  by  parameter  rate  A,  (an  event  per  unit  time).  Consider 
time  to  be  discretized  in  At  increments,  utilizing  the  underlying  properties  of  the 
geometric  distribution  as  the  arrival  process;  the  probability  that  the  target  arrives  at  time 
t  is  A,At.  Each  arrival  is  independent  from  the  other  and  memoryless  due  to  the  geometric 
distribution. 

Suppose  that  the  total  number  of  vehicles,  N,  will  not  exceed  three  vehicles  on  the 
road  network;  thus,  N  =  3.  This  can  be  assumed  due  to  a  coordinated  checkpoint  system 
in  the  road  network  to  ensure  that  the  network  is  not  overwhelmed  with  traffic.  The 
scenario  evaluates  the  arrival  of  the  objects  under  three  distinct  instances  of  inter- arrival 
rate:  A,i,  X2,  ^3  where  1  =  A,i<  A,2  <  X3. 

When  stating  that  the  arrival  of  objects  has  a  one-time  step  inter-arrival  time,  this 
simply  states  that  objects  arrive  with  one  time  unit  separation.  The  investigation  of  Blue’s 
ability  to  properly  interdict  the  target  while  observing  all  objects,  threat  and  neutral  alike, 
is  a  difficult  task.  Analyzing  the  one  inter-arrival  instance  provides  a  worse  case  basis 
against  which  all  later  scenarios  may  be  evaluated.  Then  the  thesis  will  explore  the  best 
Blue  Eorce  case  with  a  large  inter-arrival  time. 

An  Intermediate  inter-arrival  time  (X2)  is  simulated  to  ensure  that  nominal  traffic 
patterns  are  explored.  Through  simulation  and  analysis,  this  inter-arrival  time  provides 
insight  into  a  more  realistic  scenario  setting,  since  the  objects  will  not  be  all  arriving  at  a 
single  time,  nor  arriving  in  such  a  manner  that  only  a  single  object  is  in  the  scenario 
during  any  given  time. 
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The  largest  inter-arrival  time  (Xj)  utilized  for  this  thesis  is  setting  the  inter-arrival 
time  to  the  time  required  to  traverse  the  longest  road  by  any  single  object.  Under  this  rule, 
the  simulation  may  be  thought  of  as  only  exploring  a  single  object’s  motion  from  the 
entry  into  the  theater  to  the  destination  at  any  given  time.  Since  there  is  only  a  single 
object  to  be  identified  and  interdicted,  an  upper  bound  with  respect  to  the  Blue  Forces’ 
ability  to  perform  its  operations  will  be  analyzed. 

2.  Random  Selection  of  Roads 

Given  the  arrival  rate  of  vehicles  to  the  road  network,  the  avenue  of  approach,  i.e., 
the  road  traversed  by  each  vehicle,  is  also  assumed  a  random  component  of  the  scenario. 
The  theater  consists  of  three  roads  that  lead  to  the  FOB  location.  The  three  roads  have  a 
distinct  entrance  node  for  the  enemy  to  begin  its  transit.  Supposing  that  there  exist  three 
objects,  one  being  a  true  target,  the  initialization  of  where  the  objects  begin  may  be 
simulated  in  the  following  ways: 

•  The  objects  begin  on  each  of  the  three  roads,  such  that  each  road  contains 
only  one  object 

•  The  choice  of  roads  is  independent  from  object  to  object 

The  first  proposed  concept  for  object  placement  is  realized  by  drawing  a  random 
number  based  on  the  number  of  roads  that  exist.  Once  the  number  is  drawn,  the  road  is 
assigned  to  an  object  and  is  then  discarded  from  the  possible  choices  of  roads.  This 
process  is  repeated  until  all  roads  have  been  assigned  to  the  objects.  For  example, 
consider  the  Camp  Roberts’  network  where  entry  nodes  are  labeled  7,  19,  31,  as  shown  in 
Figure  1.  Utilizing  the  idea  of  assigning  a  road  to  an  object  and  then  discarding  that  road 
as  a  possible  choice  ensures  that  the  roads  are  not  duplicated  from  object  to  object. 

7?  ~  U(0,N);  where  N  is  number  of  entry  nodes 

Next  iteration: 

Ji~U(0,N-  1);  where  N  is  number  of  entry  nodes 
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The  second  concept  allows  each  object  access  to  all  of  the  entry  nodes  as  their 
initial  starting  location.  A  random  number  is  drawn  from  the  list  of  possible  starting 
locations.  Based  on  the  random  number,  the  node  is  chosen  and  then  assigned  to  an 
object.  The  iteration  of  this  process  is  continued  until  all  objects  have  an  assigned  entry 
node. 

R  ~  17(0,  A);  where  N  is  number  of  entry  nodes 

The  former  method  of  road  selection  by  the  objects  depends  on  the  objects  that 
have  already  arrived.  This  dependence  simplilies  the  scenario  because  once  an  object  has 
been  observed  on  a  single  road  and  interdicted,  then  that  road  would  not  need  to  be 
observed  again.  Although  this  is  an  assumption  of  the  arrival  process,  the  simulation  does 
not  account  for  this  dependence  and  continues  to  search  for  objects  regardless  of  whether 
an  object  was  found  on  a  single  road  or  not. 

The  road  selection  method  that  allows  for  multiple  objects  to  arrive  on  a  single 
road  is  independent  from  one  road  selection  to  the  next.  This  type  of  problem  is  more 
complex  for  interdiction  operations  due  to  the  fact  that  objects  may  arrive  on  the  same 
roads  shortly  after  other  prior  arrivals.  Therefore,  if  an  observation  is  made  and  the 
surveillance  asset  continues  on  to  the  next  road,  the  remaining  objects  may  traverse  the 
road  undetected  by  the  surveillance  asset.  Thus,  the  only  chance  that  the  object  is 
interdicted  prior  to  reaching  the  FOB  rests  with  the  surveillance  capabilities  of  the  QRF. 
Supposing  that  the  QRF  does  not  have  the  ability  to  look  out  past  the  current  location  and 
the  QRF  is  not  performing  a  patrol,  then  all  the  objects  that  leak  through  the  surveillance 
asset  will  reach  the  FOB.  Comparison  of  these  two  approaches  highlights  any  advantage 
of  considering  the  more  difficult  independent  case.  Modeling  the  interdiction  failure  and 
the  percentage  of  objects  that  reach  the  FOB  under  this  premise  provides  insight  into 
operational  tactics  and  utilization  of  the  surveillance  and  interdiction  assets. 

3,  Object  Motion  Model 

When  developing  the  SASIO  model,  the  discrete  time  step  used  to  measure  the 
progression  of  entities  represents  an  arbitrary  unit  of  time.  The  time  step  utilized 
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throughout  the  model  is  sealed  in  sueh  a  way  that  all  objeets,  neutral  or  threat,  are 
assumed  to  not  traverse  more  than  a  single  eell  in  a  given  unit  of  time.  This  allows  for  an 
observation  to  be  made  in  any  given  eell  sueh  that  the  objeet  is  either  present  or  not  due 
to  motion  through  the  eell  and  not  by  “jumping  over”  a  eell.  This  assumption  is  relevant 
and  is  utilized  when  the  surveillanee  and  QRF  assets  perform  observations  whieh  later 
feed  into  an  updated  probability  map  whieh  illustrate  the  probabilities  of  where  an  objeet 
is  within  the  theater. 

Assume  that  eaeh  vehiele  approaehes  the  FOB  with  a  eonstant,  deterministie 
speed,  denoted  by  v.  For  the  diseretized  road  network,  the  vehieles’  speeds  are  eonverted 
from  a  eontinuous  veloeity  to  that  whieh  is  diserete  whieh  eorresponds  to  the  number  of 
eells  traversed  in  a  given  time  step.  For  example,  v  =  1  eorresponds  to  a  traversal  of  one 
nodes  in  a  single  time  step,  where  as  v  =  0.25  is  analogous  to  a  slower  vehiele  requiring 
four  time  steps  to  transit  to  an  adjaeent  node.  Upon  starting  the  simulation,  the  objeets 
speeds  are  set  to  one  of  the  following:  1  (the  objeets  move  every  time  step),  2  (the  objeets 
move  in  two  time  steps,  i.e.,  v  =  0.50),  and  3  (three  time  steps  to  move,  i.e.,  v  =  0.33). 
The  justifieation  for  modeling  objeet  movement  in  this  way  follows. 

Given  the  theater  is  eonstrueted  so  all  objeet  traffie  tends  to  flow  from  an  entry 
node  towards  the  FOB,  a  direeted  graph  may  be  eonstrueted  based  on  the  adjaeeney  list 
of  the  theater.  The  pseudo  eode  in  Figure  2  represents  the  objeet  motion  model  for  the 
objeets.  From  this  implementation,  a  visual  representation  of  the  motion  model,  as  in 
Figure  3,  illustrates  the  pseudo  eode  exeeution  of  an  objeet  moving  in  3  time  steps,  as  it 
pertains  to  the  Camp  Roberts’  seenario. 
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alEodthm  .iLQblect-Wqtjpn  iVl^MJel 

Input:  DirSclEd  graph  fr'om  Theater 

□bjed's  spEEd 

Owtputi_ Dblert':i  n&jtt.lpcaeipn 


lf<n-|ALIZE; 

-Diretted  Graphs 
G' 

■^Set  C-nuntEr  =  Obj&tt  speed 

AlEprrthm 

1:  WhMe(truE] 

2:  ■‘[>EcremiEnl  Counter 

3:  5f(Counter  ==  0) 

4  -Determine  adjatenl  node 

5:  -TranSil 

B:  -Reset  Counter  10  ObjecL's  speed 

7:  End 

£; EM 

Figure  2;  Pseudo  code  representing  Object  Motion  Model  implementation 


Figure  3:  Object  motion  model  implementation  for  an  object  that  traverses  from 

node  to  node  in  three  time  steps 
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The  above  models  pertain  to  the  objeets  defined  in  the  seenario.  Future  studies 
may  investigate  more  sophistieated  eonstruetions  faeilitated  by  the  modular  design  of  the 
SASIO  framework. 

D,  DESCRIPTION  OF  THE  TASK  FORCE 

1,  Platform 

a.  Surveillance  Asset:  Unmanned  Aerial  Vehicle 

The  Unmanned  Aerial  Vehiele  (UAV)  performs  surveillanee  operations  as 
dietated  by  the  operator.  For  this  seenario,  the  UAV  performs  a  patrol  pattern  of  given 
radius  from  the  FOB.  Based  on  its  operating  speed,  the  UAV  flies  eaeh  node  in  its  given 
waypoint  list,  performing  observations  and  identifieations  upon  arrival  to  a  node, 
loitering  briefly,  and  continuing  its  patrol  route.  Such  a  pattern  can  be  easily 
accommodated  by  current  waypoint-based  UAV  auto-pilot,  enabling  operators  to  focus 
on  examining  down  linked  imagery. 

b.  Interdiction  Asset:  Quick  Reaction  Force 

The  Quick  Reaction  Force  is  a  mounted  vehicle  asset  whose  primary  role 
is  the  security  of  the  FOB.  This  asset  performs  surveillance  through  patrol  routes,  as  well 
as  interdict  potential  threats  that  are  obtained  from  the  surveillance  asset.  While 
performing  patrols,  the  QRF  performs  a  predefined  patrol  pattern  until  the  surveillance 
asset  makes  an  interdiction  request.  The  QRF  transitions  to  an  interdict  mode  and 
services  the  request  by  traversing  towards  the  location  where  the  request  originated.  Once 
an  interdiction  is  made,  the  QRF  clears  the  object  and  then  returns  back  to  the  FOB  to 
continue  its  patrol.  This  process  of  patrolling,  interdicting  and  returning  back  to  patrolling 
provides  protection  for  the  FOB  through  surveillance  and  interdicts  the  threat  while 
maintaining  a  standoff  range  from  the  FOB. 
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2. 


Probabilistic  Modeling  of  Situational  Awareness 


a.  Probability  Maps 

The  SASIO  simulation  utilizes  a  dynamie  probability  map  to  govern  a 
aetions  by  the  QRF.  A  probability  map,  as  defined  in  the  eontext  of  this  thesis,  is  an  array 
of  probabilities  for  eaeh  node  within  the  theater,  whieh  allows  the  user  to  pereeive  the 
likelihoods  of  where  an  enemy  may  be  loeated.  The  user  initializes  the  simulation  with 
what  is  believed  to  be  the  loeation  of  the  threat  or  the  probability  that  the  threat  may 
traverse  a  given  set  of  node  based  on  intelligenee  gathered  for  the  area. 

The  probability  map  representation  used  in  this  thesis  is  an  important 
aspeet  of  SI  problems.  The  probability  map’s  primary  funetion  is  to  resolve  whether  a 
query  and/or  interdietion  of  an  objeet  must  oeeur  by  the  QRF.  The  QRF  may  be 
performing  surveillanee  operations  in  eonjunetion  with  the  UAV;  however,  onee  the 
Commander  deems  that  there  is  a  threat  objeet  along  a  road  with  high  eonfidenee,  then 
the  QRF  transitions  to  the  interdietion  mode  to  pursue  sueh  a  threat.  This  transition 
direetly  relies  on  the  probability  map  rapidly  evolving  as  observations  are  made  by  the 
UAV.  If  no  intelligenee  is  obtained  for  the  AOI,  then  the  user  ean  define  a  uniform 
probability  along  the  entire  theater  as  follows 

Let  X  e  {1,  2,  ..,C}  be  defined  as  a  RV  of  the  loeation  of  the  objeet 

where  C  is  the  number  max  number  of  nodes  in  the  theater 
P[X^]  =  probability  that  a  threat  is  loeated  in  eell  C 

R[AJ~U(0,1)=  P[XJ  =  ^ 

As  the  simulation  progresses  and  the  surveillanee  asset  makes 
observations,  the  probability  map  is  updated  based  on  these  observations  aeeording  to  the 
observation  models  deseribed  in  the  following  seetion.  Suppose  that  the  surveillanee  asset 
performs  an  observation  in  eell  five  and  the  surveillanee  asset  does  not  observe  a  threat, 
then  the  probability  that  a  threat  is  in  eell  five  is  deereased  for  the  probability  map.  The 
eonverse  is  also  true.  The  way  in  whieh  the  probability  map  ineorporates  imperfeet 
observations  is  through  a  Bayesian  update. 
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b.  Bayesian  Updating 


Each  node  is  updated  independent  from  the  other  cells.  Therefore,  if  an 
observation  is  made  in  one  eell,  then  that  eell  alone  will  be  updated.  Using  this  method  to 
update  the  eells  is  used  over  a  joint  eell  updating  beeause  as  more  UAVs  are  used,  the 
updating  proeess  beeomes  more  diffieult.  Given  that  the  eells’  identifieations  are 
imperfeet,  the  single  eell  updating  is  more  plausible  beeause  after  an  observation  and 
identifieation  is  taken,  then  only  that  eell  should  be  updated. 

The  deteetion  proeess  for  the  UAV  relies  its  sensor  eharaeteristies. 
Assuming  that  the  UAV  is  equipped  with  perfeet  deteetion  sensors,  and  the  simulation 
was  run  holding  all  other  parameters  eonstant,  then  the  result  of  this  simulation  run  would 
be  the  best-ease  seenario.  Suppose  that  the  UAV  is  performing  surveillanee  in  aoeordanee 
with  a  user-defined  route.  As  the  UAV  observes  a  eell,  the  UAV  proeesses  whether  an 
objeet  is  present  or  not  with  100  pereent  eonfidenee  due  to  the  perfeet  sensor  suite. 
Therefore  only  a  single  “look”  is  required  to  verify  if  an  objeet  is  present  or  not.  Onee  an 
objeet  is  identified  in  the  eell,  then  the  elassifieation  of  whether  that  objeet  is  a  target  or 
neutral  must  oeeur.  Therefore,  if  the  UAV  could  detect  and  classify  the  objeets  in  the 
AOI  with  perfeet  eonfidenee,  then  an  upper  bound  on  the  problem  is  defined  assuming 
that  all  other  parameters  are  held  eonstant  from  an  exeeution  of  the  simulation  to  the  next. 
Sinee  teehnologieal  advanee  have  not  been  made  to  ereate  the  “perfeet-sensor  paekage,” 
this  thesis  explores  Bayesian  updating  of  the  probability  map  to  elassify  objeets  entering 
into  the  AOI  given  an  imperfeet  sensor  paekage. 

(1).  Deteetion.  The  SASIO  simulation  models  the  deteetion 
proeess  made  by  the  surveillanee  and  interdietion  assets  as  a  perfeet  sensor.  Therefore,  if 
either  asset  observes  an  objeet  in  the  eell  that  surveillanee  is  being  performed,  then  the 
asset  deteets  that  objeet.  The  SASIO  framework  does  allow  for  imperfeet  deteetion,  but 
for  the  purposes  of  this  seenario,  perfeet  deteetion  is  modeled. 

The  basis  for  the  perfeet  sensor  for  the  UAV  and  QRF  is  due  to  the 
verifieation  runs  that  were  performed  at  Camp  Roberts  in  August  of  2009.  Utilizing  a 
UAV  for  surveillanee  and  observing  the  live-feed  broadeasted  baek  to  the  user,  deteetion 
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of  an  object  was  easily  achieved  by  the  operator.  Therefore,  the  process  of  perfect 
detection  was  modeled  in  the  SASIO  simulation. 

(2).  Identification.  Since  imperfect  identification  is  modeled  in 
the  SASIO  simulation,  some  meaning  of  an  imperfect  sensor  must  be  understood.  There 
are  essentially  four  different  cases  for  probability  of  identification  that  may  occur  when 
an  observation  is  performed.  Prior  to  stating  the  four  cases,  some  nomenclature  for  this 
derivation  must  be  defined. 


Assume  that  a  positive  detection  occurs  in  cell  c;  otherwise,  no 
identification  takes  place 


as  defined  below 


Let  Zc  be  a  Bernoulli  random  variable  describing  the  true  identity 


object  type  is  a  neutral 
object  type  is  a  threat 


Let  Wc  be  a  Bernoulli  random  variable  describing  the  observation, 

as  defined  below 


^  ^  I  0,  observation  "says"  object  is  a  neutral 

1,  observation  "says"  object  is  a  threat 

There  are  four  distinct  cases  that  an  identification  can  take  on:  the 
probability  of  a  correct  negative  identification  “l-y”  (eqn.  2.1)  (correctly  observing  no 
threat  present),  the  probability  of  a  false  positive  detection  “y”  (eqn.  2.2)  (saying  the 
object  is  a  threat  when  its  not),  the  probability  of  a  missed  detection  “p”  (eqn.  2.3) 
(saying  the  object  is  a  neutral  when  it  is  a  threat),  and  finally  the  probability  of  a  correctly 
identifying  a  threat  “1-p”  (eqn.  2.4). 
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Let  =  0  represent  an  objeet  of  type  neutral  is  deteeted  in  eell  e 


=  0  represent  an  identifieation  "says"  objeet  is  a  neutral  in  eell  e 
P(VL,.  -  0  I  Z^.  -  0)-l-y  (eqn.2.1) 

=  I  represent  an  identifieation  "says"  object  is  a  threat  in  cell  c 
P(VL^,  =  1  I  Z.,  =  0)-y  (eqn.2.2) 

Let  Z^  =  1  represent  an  object  of  type  threat  is  detected  in  cell  c 

=  0  represent  an  identification  "says"  object  is  a  neutral  in  cell  c 

P(W,,  =  0  I  Z^  =  l)-p  (eqn.2.3) 

=  \  represent  an  identification  "says"  object  is  a  threat  in  cell  c 

P(VL,,  =  1  I  Z.,  =  l)-l-p  (eqn.2.4) 

So,  now  that  identification  can  be  made  utilizing  an  imperfect 
sensor,  the  probability  of  an  object  being  a  threat  based  on  prior  knowledge  of  that 
location  must  be  also  taken  into  account.  Using  this  method  of  prior  knowledge  of  the 
threat  location  in  conjunction  with  observations  being  made  by  the  surveillance  asset  will 
either  increase  the  probability  that  a  threat  is  in  the  observed  cell  or  decrease  that 
probability. 

SASIO  assumes  an  independent  cell  Bayesian  update.  This  method 
assumes  that  given  an  observation  in  a  single  cell;  the  probability  in  that  cell  alone  will 
either  increase  or  decrease  based  on  that  observation.  Other  methods  could  be  utilized, 
such  as  an  update  method,  considering  the  joint  distribution  in  which  e.g.,  a  positive 
observation  in  a  single  cell  is  made,  then  the  probability  that  the  object  in  the  other  cells 
decrease.  This  method  becomes  more  complex  if  more  than  a  single  object  exists  in  the 
AOI. 
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The  process  of  taking  observations  and  updating  the  probability  of 
where  an  object  is  located  is  an  iterative  process.  Thus,  the  prior  probability  is 
continually  updated  as  observations  are  performed.  This  updating  is  through  utilizing 
Bayes  rule  (eqn.  2.5) 


P[Z^  =zjfV^=w]  = 


F[fr^=wlZ^  =  z]*F[Z^  =  l] 
F[W^  =  w] 


(eqn.  2.5) 


The  left  hand  side  is  the  posterior  probability  that  an  object  is  a 
threat  or  neutral  given  that  an  observation  occurs  in  cell  c.  Since  in  the  presented  model, 
we  utilize  the  independent  cell-updating  rule,  updating  any  cell  other  than  the  cell  that 
observation  is  being  taken  in  does  not  need  to  be  performed.  This  decoupling  allows  for 
efficient  computations,  linear  in  the  number  of  observations  vice  number  of  cells.  On  the 
right  hand  side  of  the  equation,  the  numerator  represents  the  probability  of  an  observation 
multiplied  by  the  prior  belief  of  the  location  of  the  target.  The  denominator  is  the 
marginal  probability  of  all  observations.  As  an  example,  suppose  that  the  threat  is 
observed  in  cell  c  and  that  the  object  is  a  threat  (i.e.,  Zc=l  |  =  1).  Then  the  following 

result  for  cell  c  in  the  probability  map  occurs: 


P|Z,,  =  1IW,  =  1|  = 


_ =  1IZ,  =  1I]*P[Z,  =  1] _ 

PHV,  =  1IZ,  =  1]*P[Z,  =  1]  +  ;>[W^  =  1IZ,  =  01*(1-P[Z,  =  11) 


(eqn.  2.6) 


Substituting  eqn.  2.4  for  the  numerator  and  expanding  the  denominator  is  as  follows: 


P|Z,  =  1IIV,  =  1| 


_ (1  p)*P|Z..  =  l| _ 

(l-p)*PlZ,  =  l]  +  }'*{l-P[Z,  =  ll) 


(eqn.  2.7) 


This  iterative  process  occurs  with  each  observation  by  the  UAV.  As  the  UAV  performs 
more  and  more  observation,  the  probability  in  that  cell  continues  to  change.  Supposing 
that  the  UAV  contains  a  perfect  sensor  package,  y  and  p  are  equal  to  zero,  that  is,  no 
errors  in  identification,  then  the  equation  simplifies  to  the  following: 
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(eqn.  2.8) 


P[Z..  =  1HV,  =  1]  = 


(l)*prior  probability  in  cell  c 
(l)*prior  probability  +  0»(1  —  prior  probability  in  cell  c) 


Therefore,  if  there  were  an  ability  to  create  a  “perfect  sensor,”  then  only  a  single  positive 
identification  would  be  needed  to  locate  the  object.  However,  the  costs  to  create  an 
almost  perfect  sensor  may  be  so  extreme  that  utilizing  more  surveillance  assets  to  obtain 
more  observations  may  be  more  effective.  Understanding  the  effect  of  improving  sensors 
and  the  implied  tradeoffs  requires  a  sensitivity  analysis,  which  this  thesis  aims  to  provide. 

c.  Observation  Modeling  for  the  Quick  Reaction  Force 

The  QRF  can  perform  observations  the  same  manner  as  the  surveillance 
asset.  If  the  QRF  has  the  ability  to  observe  nodes  that  are  adjacent  to  its  current  location, 
then  observations  may  be  performed  on  these  locations.  This  may  be  thought  of  as  the 
QRF  having  a  person  peering  through  binoculars  as  the  driver  transits  the  patrol  route. 
The  lookout  may  see  a  single  node  away  (250  m  based  on  the  theater)  or  multiple  nodes 
away  depending  on  the  line  of  sight  and  focal  range  of  the  binoculars. 

SASIO  takes  advantage  of  a  lookout  when  the  QRF  is  in  a  search  mode. 
However,  the  difference  between  the  QRF  and  the  UAV  asset  is  there  are  no  detection 
and  identification  errors  associated  with  the  QRF,  while  the  UAV  has  identification 
errors.  Thus,  Bayesian  updating  updates  the  probability  map  based  on  the  UAV’s 
observations  and  aids  the  QRF  as  a  decision-support  tool  dictating  when  an  interdiction 
must  be  performed. 

d.  Report  List  Generation 

The  data  obtained  from  observations  that  result  in  a  positive  identification 
must  be  relayed  to  the  Commander  for  review  and/or  action.  This  information  is 
transmitted  in  a  report  list  format.  A  report  list  is  a  list  of  all  reports  made  by  the 
surveillance  asset.  The  report  includes  pertinent  information  regarding  the  observation 
such  as:  the  time  that  the  observation  occurred,  the  location  of  the  observation,  which 

surveillance  asset  made  the  report  and  any  description  of  the  suspected  target.  As  with  a 
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real  SI  operation  in  whieh  observations  and  reports  are  sent  to  the  Commander,  this 
report  list  proeess  is  emulated  in  the  simulation.  Action  to  either  interdict  or  query  the 
objects  on  the  report  list  represents  a  Commander  sending  out  a  QRF  to  investigate  a 
potential  threat.  SASIO  uses  the  report  list  as  a  tool  to  transition  the  QRF  from  a 
searching  phase  to  an  interdiction  phase  and  may  do  so  once  the  report  is  obtained.  Once 
the  report  is  cleared  by  the  QRF,  the  report  is  removed  from  the  list.  Current  SASIO 
model  implementation  services  the  reports  in  a  First- in.  First-out  (FIFO)  type  queue. 

3,  Trajectory  Modeling 

a.  Unmanned  Aerial  Vehicle 

The  UAV’s  patrol  path  is  not  geographically  constrained  since  its  flight 
path  is  not  limited  (other  than  by  airspace  restrictions)  for  this  scenario.  Therefore,  the 
flight  paths  that  will  be  investigated  within  this  thesis  will  be  of  varying  radii  of  250  m, 
500  m,  and  1000  m  away  from  the  FOB.  Varying  the  radius  of  the  flight  path  can  either 
aid  the  Commander  in  interdicting  threats  by  detecting  and  observing  the  objects  at  a 
longer  range;  however,  it  may  also  hinder  the  operations  due  to  the  increased  leg 
distances  from  one  waypoint  to  the  next. 


23 


Figure  4:  Varying  radii  flight  paths  for  the  UAV  with  predefined  flight  profile 

(orange)  based  on  750  m  radius  from  the  FOB 

Let  WP  be  a  list  of  waypoints  that  are  desired  for  surveillance  operations. 
Each  waypoint  distance  is  calculated  based  on  Latitude  and  Longitude  coordinates  from 
the  current  node  to  the  next  node  in  the  list.  Assuming  that  the  UAV  flies  at  a  calibrated 
speed  of  V  =  450  m/min  which  is  equivalent  to  15  knots,  the  UAV  reaches  the  next  node 
in  the  list  when  the  allotted  time  to  traverse  to  the  next  cell  is  met. 

WP={1,2,3,4} 

Let  UAV[0]  e  WP  be  defined  as  the  current  cell  of  the  UAV 

Let  UAV[1]  G  WP  be  defined  as  the  next  node  in  the  list  of  WP 

Dist{UAV[l])-Dist{UAV[0] 

Time  to  next  WP  = - 

V 

Upon  the  UAV’s  arrival  to  a  node,  the  UAV  performs  surveillance 
operations  as  described  previously.  Upon  completion  of  the  observation,  the  UAV 
continues  on  to  its  next  location.  The  user  may  desire  the  UAV  loiters  at  each  location  for 
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a  desired  time  prior  to  moving  on  to  the  next  loeation.  SASIO  model  implementation  may 
take  this  into  aeeount  to  allow  for  a  longer  “look”  at  eaeh  node  in  question.  Follow-on 
studies  to  this  thesis  may  ineorporate  a  dynamie  flight  profile  in  whieh  the  UAV  loiters  at 
a  loeation  only  if  deteetion  is  made.  This  allows  for  additional  proeessing  time  to  diseem 
the  deteetion  as  either  a  threat  or  neutral.  The  next  layer  of  diffieulty  would  inelude 
allowing  a  dynamie  flight  profile  for  the  UAV  sueh  that  if  an  objeet  is  deteeted  along  the 
road,  the  UAV  eontinues  to  traek  this  objeet  until  a  QRF  asset  ean  query  the  objeet.  Onee 
the  QRF  reaehes  the  objeet,  the  UAV  eontinues  on  with  the  predefined  patrol  path. 

b.  Quick  Reaction  Force 

(1)  Patrolling  methods.  There  are  some  interesting  methods  of 
patrolling  that  SASIO  ean  implement.  Due  to  the  modularity  of  the  simulation  model,  the 
user  ean  define  two  distinet  types  of  patrols  performed  by  the  QRF.  Stationary  or 
trajeetory-based  patrols  eurrently  exist  and  eaeh  is  analyzed  in  order  to  provide  taetieal 
insight  to  Commanders  on  strengths  and  weaknesses  of  implementing  one  method  over 
another.  The  QRF  patrols  the  AOI  when  the  report  list  is  empty.  The  SASIO  simulation 
studies  two  distinet  eases  of  patrol  that  are  eurrently  implemented,  stationary  and  a 
trajeetory  patrol  based  on  a  speeifie  range  from  the  FOB. 

Suppose  that  a  stationary  patrol  is  implemented  and  the  QRF  does 
not  have  the  ability  to  perform  observations  outside  its  eurrent  loeation,  then  the  only  line 
of  defense  for  proteetion  of  the  FOB  relies  on  the  UAV  alone.  If  the  UAV  fails  to  deteet 
and  identify  the  threat  then  the  objeet  reaehes  the  FOB  uneheeked  by  the  QRF.  Thus,  the 
QRF  must  be  enhaneed  with  the  ability  to  seout  the  area  surrounding  the  FOB  in  order  to 
provide  a  seeond  line  of  defense  for  SI  operations. 

The  seeond  method,  trajeetory  based,  is  a  patrol  by  the  QRF  in  its 
truest  sense.  Figure  5  illustrates  a  trajeetory  patrol  of  the  QRF  750  m  away  from  the  FOB. 
The  result  of  performing  a  patrol  under  this  eontext  may  provide  undesirable 
eonsequenees  that  are  eounter  intuitive  initially.  Suppose  that  the  QRF  is  performing  a 
patrol  on  the  southern  leg  of  the  theater,  there  is  a  ehanee  that  the  threat  is  driving  down 
one  of  the  other  roads  during  this  time.  Based  on  this  instanee,  the  FOB  remains 
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vulnerable  to  an  attaek.  However,  if  the  QRF  is  patrolling  the  leg  by  whieh  the  threat  is 
eoming,  then  the  QRF  performs  the  interdiction  and  thwart  any  attack  prior  to  the  threat 
ever  reaching  the  FOB. 


Figure  5;  Illustration  of  a  patrol  of  one  nodes  out  for  the  QRF 

The  final  aspect  of  the  SASIO  model  is  the  operation  of  the  QRF 
during  an  interdiction  operation.  SASIO  implements  a  Return  To  FOB  (RTB)  method 
that  utilizes  Dijkstra’s  algorithm  in  order  to  direct  the  QRF  to  the  FOB.  Dijkstra’s 
algorithm  is  utilized  to  allow  for  more  complex  scenarios.  The  rational  for  implementing 
a  RTB  operation  is  to  emulate  the  actions  that  a  Commander  would  expect  following 
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interdicting  a  potential  threat  rather  then  continuing  on  with  the  patrol.  This  allows  for  a 
reset  of  forces  to  counteract  other  threats  that  are  unknown  in  the  area. 

(2)  Service  time  for  identification  of  an  object.  The  QRF 
performs  the  interdiction  of  potential  threats  that  were  either  relayed  via  the  reports 
obtained  from  the  UAV,  or  by  visual  detection  of  these  threats  by  scouting  techniques. 
The  goal  of  the  QRF  is  to  query  and/or  interdict  the  objects  at  a  distance  away  from  the 
FOB  in  hopes  to  minimize  the  number  of  casualties  that  would  occur  in  the  event  that  the 
VBIED  reached  its  goal.  Since  the  QRF  is  a  manned  asset,  the  ability  to  identify  and 
service  the  threats  upon  a  successful  interdiction  requires  time,  which  is  a  factor  that  can 
be  varied  based  on  user  input. 

The  service  time  for  proper  identification  is  what  may  be  varied  to 
simulate  true  interdiction  operations.  Current  implementation  allows  for  this  service  time 
to  be  varied  in  constant  time  increments  from  run  to  run.  This  parameter  provides  insight 
into  the  interdiction  operations  and  an  upper  limit  on  the  service  time  to  ensure  that  the 
FOB  remains  unharmed.  By  setting  the  service  time  to  instantaneous,  time  to  service 
equal  zero,  an  upper  bound  on  the  number  of  objects  interdicted  can  be  proposed.  Future 
revisions  of  the  SASIO  model  may  incorporate  a  queuing  model  implementation 
resulting  in  a  nondeterministic  service  time. 

4.  Prediction  Model 

Gathering  intelligence  with  respect  to  the  AOI  is  one  of  the  most  vital  actions  that 
must  be  performed  immediately  by  the  Commander  and  his  personnel.  The  SASIO  model 
implements  intelligence  gathered  for  an  AOI  in  the  form  of  a  prediction  model.  This 
prediction  model  focuses  on  the  motion  of  the  enemy  force  to  include  trajectories  and 
speed  at  which  each  object  moves.  This  prediction  model  takes  the  form  of  four  different 
Markov  transition  matrices  depending  on  the  accuracy  of  the  intelligence  gathered.  Since 
this  scenario  ensures  that  all  objects  travel  from  an  entry  node  towards  the  FOB,  a 
directed  graph  is  used  to  represent  the  trajectories.  However,  the  speed  at  which  the 
objects  travel  is  unknown  to  the  Blue  Force.  There  are  three  different  speeds  at  which  the 
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objects  can  travel  as  described  under  Object  Motion  Model:  1,  2,  and  3  and  from  these 
speeds,  the  Blue  Force  may  predict  the  object’s  next  location.  Figure  7  below  depicts  how 
the  object  moves  from  one  node  to  another. 


Initialize 
t  =  v 


Figure  6:  Event  graph  depicting  the  Object’s  motion 


Since  the  Blue  Force  intelligence  includes  the  trajectories  and  the  object’s  speed, 
then  if  the  Blue  Force  had  tracked  the  object  upon  entering  the  area  since  the  object’s 
motion  can  be  predicted  given  its  current  location  and  speed.  However,  the  UAV  is  the 
only  surveillance  asset  that  the  Blue  Force  has  to  make  observations  and  the  UAV  does 
not  loiter  at  the  entry  nodes;  thus,  a  prediction  of  where  the  object  is  going  to  be  next 
must  be  made  since  each  observation  is  a  glimpse  of  a  single  time.  Suppose  that  the 
object  travels  every  three  time  steps,  and  the  UAV  observes  the  object,  the  observation 
could  have  been  made  when  the  object  got  to  that  position,  or  after  the  object  has  been 
there  for  one  or  two  time  steps.  Since  the  UAV  takes  the  observation  and  continues  on,  a 
report  stating  that  the  object  is  in  location  X,  but  the  time  that  the  object  has  been  there 
cannot  be  determined.  Therefore,  using  this  example,  there  is  a  /a  chance  that  the  object 
stays  in  that  location  while  only  a  V3  chance  the  object  will  move.  An  example  of  the 
prediction  model  for  the  given  example  is  represented  in  Figure  7. 
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Figure  7:  Prediction  Markov  transition  matrix  based  on  an  object  transitioning  from 

one  node  to  the  next  adjacent  node  in  three  time  steps.  Recall  that  node  one  represents  the 

FOB  node 

SASIO  predicts  the  motion  of  the  objects  by  drawing  a  random  number,  and  if 
that  number  is  less  than  the  probability  that  the  object  moves,  then  the  prediction  states 
that  the  object  stays  in  that  location.  This  is  iterated  each  time  step,  adding  each  random 
number  to  itself,  until  the  object  is  predicted  to  move.  Therefore,  if  a  prediction  model  is 
based  off  of  intelligence  depicting  the  objects  to  move  every  four-time  steps,  but  in 
reality  the  objects  travel  every  time  step,  then  the  objects  arrives  at  the  FOB  quicker  then 
expected  and  the  Blue  Force  may  not  interdict  the  enemy  in  time. 

Therefore,  the  parameter  of  how  well  the  intelligence  matches  the  object’s  true 
motion  is  examined.  Through  these  variations,  insight  can  be  gained  into  the  type  of 
intelligence  obtained  and  the  importance  of  the  quality  of  that  information. 

5,  Experimental  Design 

The  SASIO  model  can  be  used  to  study  the  effect  of  many  factors  on  the  response 
variables  of  interest.  Experimental  design  can  be  used  to  efficiently  explore  the  design 
space  by  simultaneously  varying  the  input  factors,  which  can  be  analyzed  by  the  user 
through  the  graphical  user  interface  (GUI)  built  for  the  SASIO  framework. 

The  response  variables  that  are  explored  in  this  thesis  are;  number  of  threat 
objects  interdicted,  time  to  interdict  all  threats,  and  the  distance  the  threats  are  interdicted 
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with  respect  to  the  FOB.  Factor  screening  is  performed  first  in  order  to  reduce  the 
dimensionality  of  the  design  space  after  which  an  analysis  on  the  remaining  factors 
carried  out. 


a.  Factors,  Levels  and  Justification 


The  factors  and  levels,  which  are  investigated  in  this  thesis,  are  illustrated 
in  Figure  8.  Each  factor  is  a  unique  characteristic  of  the  entities  in  the  simulation  and 
each  level  tries  to  encapsulate  the  factors  that  can  change  during  military  operations  that 
utilize  a  UAV  for  surveillance.  Through  varying  the  objects  and  entities,  the  model  can 
provide  insight  regarding  employment  tactics  to  overcome  the  scenario  threats.  The 
objects  may  be  varied  as  follows: 


Entities 

Type  of  Variables 

Factors 

Levels 

Description 

Low  Level  High  Level 

Objects 

Continuous 

Velocity 

[1,3] 

Nodes  /  time  step 

Numerical 

Road  Selection 

[1,2] 

Objects  choose 
road  Independent 
from  each  other 
(2)  or  have  a 
single  object  per 
road  (1) 

Continuous 

Time  between  Arrivals 

[  1 , 40  ] 

E[Tlme  between 
Arrivals] 

Agents  (UAV) 

Continuous 

Velocity 

[  30 , 45 , 60  ] 

Flight  speed  of  the 
UAV  (kts) 

Continuous 

Gamma 

[0,0.45,0.9] 

Probabilities 

(Identification) 

Continuous 

Rho 

r  l-camma  ,  0.45  ,  l-oammal 

Probabilities 

(Identification) 

Continuous 

Loiter  Time 

[1,2] 

Time  steps  for 
loiterinc 

Numerical 

Radius 

r  250  ,  1000  1 

Meters 

Agents  (QRF) 

Continuous 

Deployment  Threshold 

[0.7,  1.0] 

If  threshold  In  any 
node  Is  exceeded 
via  Probability 

Map  updating, 
then  QRF  will 
deploy  to 
Investigate  that 
node 

Numerical 

Patrol 

[0,1] 

Nodes  away  from 
FOB 

Continuous 

Clearing  Time 

[1,2] 

Time  steps 

Numerical 

Prediction  Model 

VlxPl  VlxP2  VlxP3 
V2xP2  V2xP3 

V3xP3 

Cross  matrix  of 
object's  velocity 
and  prediction 

Figure  8:  Matrix  depicting  the  factors  and  levels  used  for  factor  screening  and 

Design  of  Experiments 
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Objects 

Velocity  -  The  object’s  velocity  is  modeled  as  a  notional  value  and  can  take  on 
one  of  three  values  and  the  bounds  on  these  values  are  described  in  Figure  8.  The  object’s 
velocity  represents  the  time  required  to  traverse  from  one  node  to  the  next  adjacent  node. 
Each  value  intends  to  simulate  different  types  of  objects  whether  they  are  foot  traffic  (3), 
vehicle  on  rough  terrain  (2),  or  vehicles  on  paved  roads  (1). 

Road  selection  -  The  objects  can  be  modeled  in  two  different  ways  with  respect  to 
the  roads  selection  process.  This  factor  is  categorical  and  can  only  take  on  one  of  two 
values,  a  single  object  per  road  or  independent  road  selection.  Modeling  the  objects  as 
one  object  per  road  is  the  simpler  technique  due  to  the  fact  that  only  a  single  object 
travels  down  each  road.  Upon  successful  observation  and  identification  of  the  threat,  the 
threat  can  be  interdicted  rapidly;  therefore  minimizing  any  threat  to  the  FOB.  Modeling 
the  objects  road  selection  as  independent  from  each  other  simulates  reality  since  each 
object  can  travel  down  any  road  and  is  not  dependent  on  the  prior  object’s  selection. 

Time  between  arrivals  -  The  arrival  rates  are  modeled  as  a  stochastic  process 
where  the  inter-arrival  times  are  exponentially  distributed.  With  a  time  between  arrivals 
of  one  time  step,  this  represents  all  objects  arriving  in  the  AOI  at  nearly  the  same  time. 
As  the  time  is  increased  to  five  and  eleven,  the  time  separating  the  arrival  of  one  object  to 
the  next  is  increased.  As  the  time  is  increased,  it  is  anticipated  that  the  ease  of  interdicting 
the  threat  increases.  Therefore  with  the  maximum  time  used,  an  upper  bound  on  how  well 
the  taskforce  can  perform  can  be  studied 
Agents  (UAV) 

Identification  -  These  factors  represent  the  sensor  characteristics  of  the  UAV.  By 
varying  the  factors  such  that  p  (Rho)  is  the  compliment  of  y  (Gamma)  (identification)  the 
lower  diagonal  of  a  detection  or  identification  matrix  may  be  evaluated.  Modeling  the 
imperfect  sensor  produces  the  same  results  from  the  upper  versus  the  lower  diagonal  and 
therefore  only  one  diagonal  must  be  analyzed. 

Loiter  time  -  This  factor  represents  the  amount  of  time  that  the  UAV  remains  at  a 
single  node  for  detection  and  identification  operations.  The  levels  that  studied  are  a  one 
and  two  time  step  loiter  time.  The  one  time  step  represents  an  observation  that  occurs  in 
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one  time  step  prior  to  moving  to  the  next  waypoint  while  the  two  time  step  represents  a 
loitering  to  allow  for  a  more  time  for  an  deteetion  and  identifieation  operations. 

Patrol  radius  -  The  UAV  flies  predetermined  routes  for  eaeh  simulation  run.  This 
faetor  explores  the  variations  in  the  response  as  the  radius  inereases  away  from  the  FOB. 
The  radii  are  measured  as  Euelidean  distanee  from  the  FOB  and  provides  insight  into 
whether  a  farther  deteetion  range  is  better  than  performing  more  observations  at  a  eloser 
distanee. 

Agents  (QRF) 

Patrol  -  The  QRF’s  patrol  studied  under  a  stationary  eontext,  as  well  as  patrolling 
a  single  node  away  from  the  FOB.  These  distanees  provide  quiek  response  if  a  report  is 
generated  or  if  a  threat  is  pereeived  to  be  elose  to  the  FOB.  As  the  patrol  distanee  is 
inereased  and  the  threat  is  traversing  toward  the  FOB  via  a  different  road,  then  the  QRF 
must  traverse  baek  to  the  FOB  and  then  interdiet  the  threat.  The  time  that  is  required  for 
the  QRF  to  return  to  the  FOB  is  additional  time  that  the  threat  has  to  reaeh  the  FOB. 
Therefore,  under  these  eireumstanees,  there  is  an  inereased  probability  that  the  threat 
reaches  the  FOB  if  the  patrol  radius  is  increased  dramatically. 

Clearing  time  -  This  factor  relates  to  the  amount  of  time  that  is  required  to 
identify  an  object  as  a  threat  or  not  once  the  object  is  interdicted.  Instantaneous  clearing 
time  takes  a  single  time  step  to  identify  the  object.  The  single-time  step  and  a  two-time 
step  delay  clearing  time  is  evaluated.  This  time  reflects  the  amount  of  time  required  to 
service  the  object  once  acquired.  As  the  time  is  increased,  the  QRF  remains  with  the 
object  longer  and  therefore  may  have  increased  difficulty  interdicting  another  object 
traversing  down  a  different  road  may  reach  the  FOB. 

Prediction  model  -  The  prediction  model  is  based  on  the  perceived  velocities  of 
the  objects  through  intelligence  gathering.  As  information  is  gathered  on  the  nominal 
traffic  patterns  and  traffic  flows  of  the  objects,  a  prediction  on  where  the  objects  are 
given  their  current  location  can  be  made.  The  prediction  model  is  a  Markov  transition 
matrix  that  is  studied  against  the  object’s  true  velocity  based  on  the  object  motion  model 
aforementioned  within  object  velocity. 
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VI  represents  an  object’s  ability  to  transition  from  one  node  to  the  next  adjacent 
node  in  one  time  step.  V2  and  V3  are  synonymous  with  two  and  three  time  steps, 
respectively,  to  transition  from  one  node  to  the  next.  The  P  represents  the  prediction 
model  as  described  above  where  PI,  P2  and  P3  depict  an  object’s  motion  based  on  of 
one,  two  or  three  time  step  transition  from  one  node  to  the  next  adjacent  node.  The 
screening  process  explores  all  combinations  of  the  velocities  and  the  prediction  models. 
These  cross  designs  illustrate  how  well  the  intelligence  gathered  represents  the  true 
motion  of  the  objects.  The  worst-case  scenario  is  the  VlxP3  or  vise  versa  model  since  the 
objects  true  motion  is  a  single  time  step  to  transition  while  the  intelligence  states  that  the 
objects  tend  to  move  every  three-time  steps.  Therefore,  utilizing  the  SASIO  model, 
analysis  on  the  importance  of  the  intelligence  gathered,  as  modeled  by  correctness  of  the 
motion  model  approximation,  can  be  explored. 

b.  Response  Variable  and  Justification 

This  thesis  explores  a  number  of  factors  and  levels,  which  are  illustrated  in 
Figure  8.  Through  varying  the  objects  and  entities,  the  model  can  simulate  the 
unpredictability  of  the  objects  and  ultimately  provide  insight  regarding  employment 
tactics  based  on  identifying  the  factors  that  have  the  most  impact  on  the  response. 

Response  variables 

Percentage  of  threats  interdicted  -  From  the  clearing  list,  the  number  of 
threats  interdicted  from  each  design  point  is  obtained  and  the  percentage  of  threats 
interdicted  is  calculated  based  on  the  number  of  replications  used  for  each  design  point. 
Through  analysis  of  this  response,  the  factors  based  on  enemy  characteristics,  as  well  as. 
Blue  Force  asset  employment  and  characteristics  can  be  obtained. 

Time  to  interdict  all  threats  -  From  the  clearing  list,  the  time  that  the 
objects  were  cleared  is  obtained.  This  response  variable  is  explored  to  determine  how 
many  threats  in  the  AOI  are  identified  and  cleared.  Through  measuring  this  response,  the 
user  can  obtain  better  employment  of  the  assets  to  maximize  the  number  of  threats 


obtained.  Along  with  the  employment  tactics,  this  response  may  provide  insight  into  the 
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worst-case  settings  and  necessity  to  request  more  assets  in  order  to  interdict  all  of  the 
potential  threats. 


Distance  the  threats  are  interdicted  with  respect  to  the  FOB  —  Since  the 
purpose  of  this  model  is  to  provide  insight  into  the  tactics  required  to  protect  the  FOB,  an 
understanding  of  the  distance  that  the  threats  are  interdicted  with  respect  to  the  FOB  must 
be  studied.  Figure  9  illustrates  these  VBIED  threats  and  their  destruction  radius  varies 
based  on  the  amount  of  explosives  that  the  threat  carries.  Supposing  that  intelligence 
provides  the  type  of  vehicles  utilized  for  VBIED  based  on  recent  attacks,  the  user  can 
explore  and/or  predict  the  maximum  the  standoff  distance  obtainable  with  current  assets 
using  the  SASIO  simulation  model.  The  analysis  accounts  for  only  the  threats  that  are 
interdicted. 


BAI'F  Explosive  Standards 


ATF 

Vehicle 

Description 

Maximum 

Explosives 

Capacity 

Lethal  Air 
Blast  Range 

Minimum 

Evacuation 

Distance 

Falling  Glass 
Hazard 

Compact 

Sedan 

5(X)  pounds 
227  KUOS 
(In  Trunk) 

100  Feet 

30  Meters 

1,500  Feet 
457  Meters 

1,250  Feet 

381  Meters 

Full  Size 
Sedan 

1,000  Pounds 
455  KUOS 
(In  Trunk) 

125  Feet 

38  Meters 

1,750  Feet 

534  Meters 

1,750  Feet 
534  Meters 

Passenger  Van 
or  Cargo  Van 

4,000  Pounds 
1,818  KUOS 

200  Feet 

61  Meters 

2,750  Feet 
838  Meters 

2,750  Feet 
838  Meters 

Small  Box  Van 
(14  Ft.  tx)x) 

10,000  Pounds 
4,545  Kilos 

300  Feet 

91  Meters 

3,750  Feet 
1,143  Meters 

3,750  Feet 
1,143  Meters 

Box  Van  or 
Water/Fuel  Truck 

30,000  Pounds 
13,636  Kilos 

450  Feet 

137  Meters 

6,500  Feet 
1,982  Meters 

6,500  Feet 
1,982  Meters 

Semi-Trailer 

60,000  Pounds 
27,273  Kilos 

600  Feet 

183  Meters 

7,000  Feet 
2,134  Meters 

7,000  Feet 
2,134  Meters 

Eigure  9;  Chart  illustrating  the  Vehiele  type  with  respect  to  the  destructive 
capabilities  based  on  the  amount  of  explosives  that  can  be  carried, 
http;//www.globalsecurity.org/military/intro/images/vbied-standards-chart.jpg 


c.  Design  of  Experiments 


Experimental  design  is  the  purposeful  control  of  the  input  factors  to  the 
experiment  to  obtain  their  relationship  (if  any)  with  the  response  variable  (Montgomery, 
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2009).  Through  utilizing  efficient  techniques,  information  collection  for  analysis 
purposes  can  be  obtained  in  the  most  cost-effective  manner.  Experimental  design  allows 
for  the  control  of  factors  through  simulation  that  would  be  difficult  to  control  in  reality. 
As  an  example,  this  simulation  allows  the  programmer  to  control  the  rate  at  which  threats 
arrive.  In  practice,  this  is  an  “uncontrollable”  factor,  but  in  the  context  of  this  simulation, 
it  can  be  systematically  changed  and  thus  provide  insights  that  would  otherwise  be 
unattainable. 

Factor  screening  is  the  process  of  systematically  varying  the  input  factors 
in  a  way  that  allows  the  identification  of  the  most  influential  with  respect  to  the 
responses.  The  output  from  the  screening  provides  the  analyst  with  the  main  effect  and 
interactions  that  are  primarily  driving  the  desired  response  variable.  The  design  matrix 
contains  12  factors  that  are  varied  throughout  the  experiment 

One  method  of  exploring  the  factor  space  is  through  a  full-factorial  design, 
which  is  neither  efficient  nor  desired  due  to  the  large  number  of  experiments  required. 
The  number  of  experiments  required  to  be  performed  utilizing  12  factors  with  2  levels  is 
2  or  4096  design  points  with  zero  replications.  However,  using  the  factor  screening 
technique  with  fractional  factorial  design  to  examine  main  effects  and  2"‘^-order 
interactions,  the  number  of  runs  can  be  reduced  to  2*^'^  where  k  is  the  number  of  factors 
and  p  is  the  number  of  generators.  Therefore  utilizing  a  fractional  factorial  design  can 
reduce  the  number  of  design  points  to  2'^'^  =  l'  =  128  design  points. 

Factorial  designs  are  an  effective  and  efficient  method  for  designing 
experiments  intended  for  factor  screening.  Fractional  factorial  designs  provide  a  nice 
alternative  to  running  full  factorial  designs  because  they  provide  information  on  main 
effects  and  low-order  interactions  in  far  fewer  experimental  tests.  Note  that  linear 
regression  is  used  to  analyze  the  results  from  factorial  designs.  The  classic  factorial 
designs  recommended  for  factor  screening  utilize  only  two  levels  of  each  input  factor  -  a 
low  level  and  a  high  level  (Montgomery,  2009). 
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III.  FACTOR  SCREENING:  NUMERICAL  ANALYSIS 


Analysis  was  performed  with  respeet  to  eaeh  response  through  a  sereening  design 
and  followed  by  eross-validation  based  on  the  influential  faetors  obtained  from  faetor 
sereening.  Often  we  are  interested  in  determining  what  faetors  affeet  a  partieular  response 
of  interest.  In  order  to  aeeomplish  this,  we  ean  design  an  experiment  that  allows  us  to 
estimate  the  magnitude  and  direetion  of  the  faetor  effeets  (sueh  as  main  effeets  and  two 
faetor  interaetions).  Faetor  effeets  deseribe  numerieally  how  mueh  the  response  ehanges 
when  eaeh  faetor  is  ehanged.  This  type  of  experiment  is  ealled  a  sereening  experiment 
(and  is  sometimes  referred  to  as  a  eharaeterization  experiment)  (Montgomery,  2009).  The 
proeess  of  sereening  the  data  for  the  three  responses:  (1)  pereentage  of  threats  interdieted, 
(2)  time  to  aequire  threats  and  (3)  distanee  with  respeet  to  the  FOB  that  the  threats  were 
interdieted,  is  deseribed  within  this  ehapter.  This  ehapter  diseusses  the  results  from  the 
sereening  experiment  for  eaeh  of  the  three  responses. 

A,  PERCENTAGE  OF  THREATS  INTERDICTED 


As  diseussed  in  Chapter  II,  eounting  the  number  of  threats  interdieted  on  a  single 
design  point  and  averaging  that  number  based  on  1000  replieations  of  that  design  point, 
the  pereentage  of  threats  interdieted  ean  be  obtained.  The  pereentage  of  threats 
interdieted  is  a  Bernoulli  random  variable,  whieh  takes  the  values,  zero  for  not  interdieted 
and  1  for  interdieted  by  Blue  Foree.  This  response  was  transformed  via  a  Log  Odds,  also 
known  as  the  “logit”  funetion,  transformation  to  ensure  the  values  remain  between  zero 
and  one  for  the  analysis  is  deseribed  below: 


Transformation  = 


,  ^  Percentage  of  threats  interdicted  ^ 

ln( - )  =  T  =x/3+s 

1  -  Percentage  of  threats  interdicted 


(eqn.  3.1) 


Standard  stepwise  regression  is  performed  in  IMP  (IMP,  2009).  The  linear 
regression  model  test  the  following  hypothesis: 

Hq  :  =  y0,2...  =  /3^  =  Q,  wherB  n  is  the  number  of  coefficients 

:  at  least  one  coefficient  ^  0  (eqn.  3.2) 
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Figure  10; 


▼  Response  Log  odds  transform  of  Percentage  of  threats 
Actual  by  Predicted  Plot 


Log  odds  transform  of  Percentage  of 
threats 

Prediaed  P<.0001  RSq-0.81 
RMSE-0.0544 


Summary  of  Fit 
RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 


Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

Model 

18 

2.3438166 

0.130212 

Error 

183 

0.5423905 

0.002964, 

C.  Total 

201 

2.8862071 

<.0001-  1 


Linear  regression  model  of  Log  Odds  transformation  of  percentage  of 


threats  interdicted 


Sorted  Parameter  Estimates 

Term 

Rho 

Object.Velocity 

Road_Seleciion(-l)*Patrol(-l] 

Rho*Object_Velocitv 

Arrival_Time 

Arrival_Time*Road_Selection[-l] 

Road_Selection(-l) 

Patrol!- 1] 

Object_Velocitv*Patrol[-l] 

UAV_Velocitv'Rho 

Arrival_Time*Patrol[-l) 

UAV_VelocitVObject_Velocity 

UAV_Velocity 

Rho*UAV_Radius 

Road_Selection(-l)*Object_Velocity 

UAV_Velocity*Loiter_Time 

Loiter_Time 

UAV_Radius 


Estimate 

Std  Error 

-0.062684 

0.004203 

0.0528697 

0.00427 

0.0295146 

0.003926 

-0.028155 

0.004331 

-0.021945 

0.003966 

0.0211875 

0.004055 

-0.018698 

0.003839 

-0.018574 

0.00388 

-0.018242 

0.003986 

-0.017577 

0.004238 

0.0164039 

0.004064 

0.0141289 

0.004 

0.0138532 

0.004252 

0.013957 

0.004295 

-0.012697 

0.003983 

0.0127783 

0.004025 

0.0120712 

0.003894 

-0.012786 

0.004315 

t  Ratio 


-14.92 

12.38 

7.52 
-6.50 
-5.53 

5.22 

-4.87 

-4.79 

-4.58 

-4.15 

4.04 

3.53 
3.26 
3.25 

-3.19 

3.17 

3.10 

-2.96 


Prob>|t| 

<.0001* 

<.0001’ 

<.0001* 

<.0001* 

<.0001* 

<.0001- 

<.0001* 

<.0001- 

<.0001* 

<.0001* 

<.0001* 

0.0005* 

0.0013* 

0.0014* 

0.0017* 

0.0018* 

0.0022* 

0.0034* 


Figure  1 1 ;  Relative  significance  of  each  factor  based  on  a  linear  regression  of  the  Log 
Odds  transformation  of  percentage  of  threats  interdicted. 


Based  on  an  F-statistic  (p-value  <  .0001),  shown  in  Figure  10,  the  null  hypothesis  is 
rejected  using  a  .1  percent  confidence  level;  therefore,  at  least  one  of  the  coefficients  in 
the  model  is  not  equal  to  zero.  Next,  the  value,  denoted  as  Rsquared,  has  a  value  of 
0.81  and  Adjusted  R^,  denoted  as  Rsquared  Adj,  has  a  value  of  0.79  as  shown  in  Figure 
10.  These  values  represent  the  amount  of  variability  in  the  response  that  is  explained  by 
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this  model  and  conclude  that  the  model  presented  is  a  satisfactory  linear  model.  Since  this 
model  is  used  to  represent  the  data  for  the  screening  process,  the  factors  that  influence 
this  response  must  be  obtained.  Figure  1 1  represents  the  relative  weight  of  each  factor 
that  influences  the  response  variable.  For  the  purposes  of  this  screening  analysis,  the  top 
20  percent  of  the  total  number  of  factors  and  interactions  is  chosen  for  the  cross- 
validation  analysis  described  in  Chapter  IV.  Given  the  value  of  R  is  high,  the  Pareto 
effect,  which  states  that  80%  of  the  results  is  due  to  20%  of  the  input,  is  the  basis  for 
choosing  only  20%  of  the  significant  factors  and  allows  for  a  minimum  number  of  factors 
used  in  modeling  to  correctly  and  satisfactorily  (neither  over-fitting  nor  under-fitting  the 
data)  describe  the  data  and  are  as  follows: 

•  Rho 

•  Object  velocity 

•  Road  selection 

•  QRF  patrol 

•  Inter- arrival  time  of  the  objects 

Upon  completion  of  analyzing  the  other  responses,  these  factors,  as  well  as  the  factors 
from  the  other  responses  is  aggregated  so  that  cross-validation  and  prediction  analysis 
can  be  performed. 

Figure  11  shows  the  relative  weightings  of  each  factor.  The  highest  weighting 
factor  is  Rho,  which  directly  relates  to  the  classification  of  the  threats  by  the  UAV.  This 
factor  has  a  negative  coefficient  and  therefore  one  can  conclude  that  as  one  decreases  this 
false  negative  identification  error  rate  in  the  sensors,  then  the  percentage  of  threats 
interdicted  increases.  That  is,  the  better  the  sensor  is  at  identification,  then  the  better  the 
chance  of  properly  identifying  the  threat  becomes  (decreasing  Rho  by  one  unit  increases 
the  identification  by  6%).  The  next  factor  is  the  object’s  velocity  as  the  objects  take 
longer  to  traverse  from  node  to  node  then  the  probability  of  interdicting  the  threat  also 
increases.  Therefore,  if  the  Blue  Force  can  perturb  the  speed  of  the  objects  in  the  AOI, 
then  there  is  a  higher  likelihood  of  interdicting  the  threat.  The  next  factor  is  the 
interaction  between  road  selection  and  patrol.  If  the  Blue  Force  can  utilize  checkpoints  to 
ensure  that  a  single  object  traverses  a  road  per  a  given  timeframe  along  with  performing 

patrols,  then  the  threat  is  more  likely  to  be  obtained. 
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B.  TIME  TO  ACQUIRE  THREAT 

The  time  to  aequire  the  threat  is  defined  as  the  time  between  a  given  threat’s 
arrival  into  the  AOI  and  its  successful  interdiction  by  the  QRF.  Using  this  calculated 
value  as  the  response,  a  best-fit  model  is  constructed  based  on  1000  replications  to 
analyze  the  influential  factors  for  this  response. 


[^Response  Time  to  acquire  threats 
*  Actual  by  Predicted  Plot 


P<.0001  RSq-0.94  RMSE- 1.8543 


Summary  of  Fit 
RSquare 
RSquare  AdJ 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

39 

8465.0773 

217.053 

63.1242 

Error 

161 

553.6004 

3.43^ 

Prob  >T 

C.  Total 

200 

9018.6777 

.  <.0001*. 

Figure  12:  Linear  regression  model  of  the  time  to  acquire  threats 


Sorted  Parameter  Estimates 

Term 

Object_Velority 

UAV_Radius 

Arrival_Time*Patrol[-l] 

UAV_Radius*Object_Velocity 

Object_Velocity*Patroll-l] 

Road_Selection[-l)*Patrol|-l] 


Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

6.0259496 

0.149718 

40.25 

<.0001* 

-1.232851 

0.140839 

-8.75 

H 

<.0001* 

0.8814253 

0.144583 

6.10 

<.0001* 

-0.792273 

0.145101 

-5.46 

[, 

<.0001* 

-0.600604 

0.144075 

-4.17 

1 

<.0001* 

0.5615126 

0.141738 

3.96 

0.0001* 

Figure  13:  The  relative  ordering  of  factors  influencing  the  time  to  acquire  threats 


Based  on  an  ANOVA  hypothesis  test,  eqn.  3.3,  the  null  hypothesis  is  rejected 
based  on  a  F-statistic  (p  <  .0001)  and  using  a  confidence  level  of  .01%. 

Hq  :/?o  =  =  y0j2...  =  =  0,  where  n  is  the  number  of  coefficients 

:  at  least  one  coefficient  ^  0  (eqn.  3.3) 
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The  =0.93  and  Adjusted  =  0.92  indicates  that  the  model  explains  most  of 
the  variability  of  the  response.  The  top  20%  factors  that  influence  the  response  are: 

•  Object  Velocity 

•  UAV  Radius 

•  QRF  Patrol 

•  Inter-arrival  Time  of  the  Objects 

Based  on  the  factor  weighting  profile,  Figure  13,  the  most  significant  factor  is 
object  velocity.  As  stated  with  percentage  threats  interdicted,  if  mechanisms  such  as 
speed  bumps  or  checkpoints  are  used  to  perturb  the  object’s  speed,  then  the  time  to 
acquire  the  threat  increases.  Also,  as  the  UAV  radius  is  decreased,  then  the  time  to 
acquire  the  threat  is  decreased.  This  is  potentially  due  to  the  UAV  identifying  more 
nearby  threats  while  performing  its  operation  and  therefore  issuing  reports  for  the  QRF  to 
interdict. 

C.  MEAN  DISTANCE  AWAY  FROM  THE  FOB 

The  mean  distance  away  from  the  FOB  that  the  threat  was  interdicted  is 
calculated  by  observing  the  average  distance  that  the  threat  was  interdicted  with  respect 
to  the  FOB  for  each  design  point  replicated  1000  times.  A  best- fit  model  was  generated 
using  this  value  as  the  response  variable  and  the  input  factors  as  the  regressors.  The 
model  for  this  analysis  can  be  seen  in  Figure  14. 
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Figure  14; 


E Response  Mean_distanceFroniFOBThreat 
Actual  by  Predicted  Plot 


Predicted 

P<.0001  RSq-0.97  RMSE-0.1857 


^  Summary  of  Fit 
RSquare 
RSquare  AdJ 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

Model 

46 

161.57780 

3.51256 

Error 

154 

5.30868 

0.03447i 

C.  Total 

200 

166.88648 

Linear  regression  model  for  mean  distance  from  FOB 


Sorted  Parameter  Estimates 

Term 

Object_Velocity 

UAV_Raclius 

Camma’Rho 

Gamma 

Rho 

Object_Velocity*  Patrol!- 1) 

Patrol[-l] 

Arrival_Time*Patrol[-ll 

Road_Selection[-l)*UAV_Radius 

UAV_Radius*Object_Velocity 

Road_SelectionI-l)*Patrol[-l| 

Arri  val_Ti  m  e  *  U  AV_Rad  i  u  s 
Road_Selection[-l)*Object_Velocity 
UAV_Radius*Patrol(-ll 
Road_Selection[-ll 


Estimate 

Std  Error 

0.6499086 

0.01561 

0.5071384 

0.015844 

2.6714134 

0.136129 

2.6558959 

0.137283 

2.6471123 

0.137175 

0.1185227 

0.014317 

0.1094159 

0.014916 

-0.099192 

0.014614 

0.0945061 

0.014652 

0.0877332 

0.014688 

-0.082889 

0.014048 

0.0817591 

0.015208 

0.071135 

0.01483 

0.0711642 

0.014863 

0.0655551 

0.01518 

t  Ratio 


Prob>|t| 

<.0001* 

<.0001‘ 

<.0001- 

<.oooi- 

<.0001' 

<.0001' 

<■0001* 

<.0001' 

<.0001' 

<.0001' 

<■0001* 

<.0001' 

<.0001' 

<.0001' 

<.0001' 


Figure  15:  The  relative  ordering  of  the  factors  influencing  the  mean  distance  away 

from  the  FOB 


The  hypothesis  testing  for  this  model  is  as  follows; 

Hq  :/7g  =  =  y0j2...  =  =  0,  whcrc  n  is  the  number  of  coefficients 

:  at  least  one  coefficient  ^  0  (equ.  3.4) 

As  before,  the  F-statistic  (p-value  <  .0001)  was  calculated,  which  means  that  the 
alternative  hypothesis  is  preferred  over  the  null.  By  observing  the  =0.97  and  the 
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Adjusted  =  0.96,  the  conclusion  that  this  model  explains  most  of  the  variability  in  the 
response  can  be  made.  Therefore,  using  this  model,  the  following  factors  are  influential  in 
the  response  variable; 

•  Object  velocity 

•  UAV  radius 

•  Gamma 

•  Rho 

•  QRF  patrol 

By  looking  at  the  weighting  profile  in  Figure  15,  Object  velocity  is  the  most 
dominate  factor.  Therefore,  like  the  other  response  variables,  if  the  object’s  speed  was 
reduced,  then  the  distance  that  the  threat  is  interdicted  is  increased.  Likewise,  if  the 
UAV’s  radius  is  increased,  the  chance  to  identify  the  threat  at  an  increased  distance 
means  that  the  threat  is  interdicted  by  the  QRF  at  an  increased  distance.  Based  on  these 
findings,  the  employment  of  using  the  UAV  at  an  increased  radius  is  more  beneficial  to 
maximizing  the  distance  to  the  FOB  then  one  with  a  smaller  radius. 

D,  CONCLUSION 

Based  on  performing  screening  for  each  response,  best-fit  models  were 
constructed.  The  models  for  each  response  had  a  high  R  value,  which  suggests  adequate 
fits  from  this  analysis,  each  best-fit  model  was  constructed  and  adequately  modeled  the 
response  variables  with  respect  to  the  factor  inputs.  The  aggregated  list  of  factors 
obtained  from  the  above  three  models  are  utilized  in  further  cross-validation  and 
prediction  analysis  discussed  in  Chapter  IV. 

The  list  of  factors  (and  associated  responses)  was  aggregated  together  so  that  each 
response  is  using  the  same  factor  set.  The  reasoning  for  this  is  to  create  a  prediction 
model  based  on  all  of  the  factors  and  to  illustrate  how  each  factor  contributes  to  the 
response  variables.  As  seen  in  Chapter  IV,  the  SASIO  model  was  run  with  intermediate 
design  points  related  to  these  factors.  Prediction  models  were  then  constructed  to  see  how 
well  these  prediction  models  correctly  represented  the  new  data  obtained  from  additional 
simulation  runs. 
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Responses 

Factors 

%  Threat 
interdicted 

Time  to 
acquire 
threats 

Distance 
from  FOB 

Object  velocity 

X 

X 

X 

UAV  radius 

X 

X 

Gamma 

X 

Rho 

X 

X 

QRF  patrol 

X 

X 

X 

Inter-arrival  time  of  objects 

X 

X 

Road  selection  by  the 
objects 

X 

Table  1 :  List  of  influential  faetors  based  on  the  response  variables 
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IV.  DESIGN  OF  EXPERIMENTS:  CROSS-VALIDATION 


A,  OVERVIEW 

New  design  points  were  generated  based  on  the  findings  of  the  sereening  exereise. 
The  levels  within  this  design  inelude  both  the  high  and  low  levels  of  the  faetors,  as  well 
as  intermediate  values,  whieh  are  then  transformed  into  eoded  units  as  in  the  sereening 
exereise  for  eross-validation.  The  design  points  (in  engineering  units)  were  then 
simulated  in  the  SASIO  model  in  order  to  explore  eaeh  response  variable  in  more  depth. 
The  predietion  model  formulated  in  the  sereening  design  is  tested  using  eross-validation 
to  see  how  well  the  predietion  model  performs. 

B,  CROSS-VALIDATION  OF  PREDICTION  DATA 

Based  on  a  fraetional  faetorial  design  of  the  input  faetors,  128  design  points  were 
ereated  and  half  were  used  in  eross-validation.  Therefore,  taking  a  subset  of  the 
predietion  design  points  (random  seleetion  of  64)  with  the  sereening  design  points,  a  new 
predietion  model  was  ereated.  Through  a  visual  representation  of  eaeh  response  with  their 
respeetive  predietion  models,  analysis  on  the  adequaey  of  the  fit  may  be  performed. 

1.  Percentage  of  Threats  Interdicted 

The  signifieant  faetors  for  eaeh  of  the  response  variables  were  used  in  ereating  the 
predietion  models.  Figure  16,  for  the  pereentage  of  threats  introdueed.  The  202 
(fraetional-faetorial  design  points  ereated  by  JMP)  design  points  from  the  sereening 
exereise  were  joined  with  a  random  seleetion  of  64  intermediate  design  points  ereated  for 
eross-validation.  This  set  of  266  design  points  were  then  used  to  develop  a  predietion 
model  based  on  the  Log  Odds  transformation  of  the  pereentage  of  threats  interdieted.  The 
predietion  equation,  along  with  a  95%  predietion  interval,  was  then  transformed  baek  to 
pereentage  of  threats  so  that  an  overlay  graph  eould  be  ereated  to  analyze  the  pereentage 
of  threats  interdieted.  Figure  17  illustrates  the  predietion  model  with  the  remaining  design 
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points.  Each  continuous  factor  has  their  associated  coefficients,  and  for  the  numerical 
factors,  the  use  of  the  Match  statement  is  used  depending  on  the  level  utilized. 


Figure  16; 


4.52 
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Prediction  model  equation  for  Log  Odds  transformation  for  percentage  of 
threats  interdieted 
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▼  B  Overlay  Plot 


Y  O  Percentage  of  Threats  in  total  Reps 

’  Prediction  of  percent  threats  interdicted 
O  Lower  Prediction  of  percent  threats  interdicted 
*  Upper  Prediction  of  percent  threats  interdicted 

A 

Figure  17;  95%  prediction  model  fit  for  the  percentage  of  threats  interdicted  with  an 

overlay  plot  of  the  data  obtained  from  a  subset  of  the  sensitivity  design  points 

Figure  17  illustrates  that  there  are  nine  data  points  that  fall  outside  the  95% 
prediction  interval,  which  equate  to  14%  of  the  cross-validation  data  points.  Analysis  of 
the  dominant  factor,  object  velocity,  indicates  that  the  data  points,  which  fall  above  the 
upper  prediction  interval  primarily  had  the  fastest  speed,  while  those  that  fell  below  the 
lower  limit  had  the  slower  speed.  From  this  insight,  sub-analysis  is  performed  to  create  a 
prediction  model  based  on  the  individual  speeds  (1,  2  or  3),  thereby  providing 
recommendation  on  employment  strategies  based  on  the  terrain  profiles  that  constrain  the 
object’s  velocity. 

2,  Time  to  Acquire  Threats 

The  set  of  266  design  points,  202  from  the  screening  exercise  and  64  from  the 
sensitivity  design  points  were  used  to  create  a  prediction  model  for  the  response  variable, 
i.e.,  time  to  acquire  threats.  This  process  of  cross-validation  was  utilized  to  achieve  a 
more  accurate  prediction  model.  Figure  18,  by  incorporating  intermediate  factor  levels 
that  were  concluded  to  be  significant  from  the  screening  design.  Based  on  this  linear 
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regression  model,  the  prediction  model  with  a  95%  prediction  interval  was  constructed. 
The  remaining  64  sensitivity  design  points  were  then  overlaid  on  this  prediction  plot  as 
shown  in  Figure  19. 
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Figure  18: 


Prediction  model  equation  for  time  to  acquire  threats 
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^  B  Overlay  Plot 


Y  O  Time  to  acquire  threats 

-t-  Pred  Formula  Time  to  acquire  threats 
O  Lower  95%  Indiv  Time  to  acquire  threats 
X  Upper  95%  Indiv  Time  to  acquire  threats 

//. 

Figure  19;  95%  prediction  model  fit  for  the  time  to  acquire  threats  with  an  overlay 

plot  of  the  data  obtained  from  a  subset  of  the  sensitivity  design  points 

Figure  19  shows  all  of  the  design  points  fall  within  the  prediction  interval  bounds. 
Based  on  this  graph,  the  prediction  model  constructed  using  the  described  technique  is  an 
adequate  prediction  model.  As  seen  in  the  graph,  there  are  three  distinct  levels  of  the  data 
points,  a  lower,  intermediate  and  an  upper  level.  Each  of  these  levels  is  due  to  the 
influence  of  the  object’s  velocity.  The  lower  segment  correlates  to  the  objects  having  a 
faster  speed  while  the  upper  segment  is  the  objects  with  the  slower  speed.  The  reasoning 
is  that  the  objects  travel  toward  the  FOB  at  a  faster  rate  and  therefore  take  a  shorter 
amount  of  time  before  being  interdicted.  Likewise,  the  objects  that  travel  toward  the  FOB 
at  a  slower  rate  are  acquired  in  a  longer  period  of  time. 

3,  Mean  Distance  to  FOB 

The  same  techniques  used  for  creating  the  prediction  model  for  the  time  to 
acquire  the  threat  response  was  used  for  mean  distance  to  FOB.  Figure  20  represents  the 
prediction  model  based  on  the  best-fit  linear  regression  model  and  the  subset  of  cross- 
validation  points  were  overlaid  on  this  prediction  model  for  further  analysis  illustrated  in 
Figure  21. 
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Figure  20:  Prediction  model  equation  for  mean  distance  to  FOB 
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▼  [^Overlay  Plot 


Y  O  Mean_disianceFromFOBThreat 

♦  Pred  Formula  Mean_distanceFromFOBThreat 
O  Lower  9556  Indiv  Mean_disianceFromFOBThreat 
Upper  9556  Indiv  Mean_distanceFromFOBThreai 

/a 

Figure  21 :  95%  prediction  model  fit  for  the  mean  distance  to  FOB  with  an  overlay 

plot  of  the  data  obtained  from  a  subset  of  the  sensitivity  design  points 

Figure  21  illustrates  that  there  are  six  data  points  that  fall  outside  the  lower 
prediction  interval,  which  equate  to  9%  of  the  64  sensitivity  design  points.  Further 
investigation  of  the  characteristics  of  these  design  points  represents  the  influence  of  the 
interaction  between  patrol  and  object  velocity.  If  the  QRF  does  not  patrol,  i.e.,  remains 
stationary  at  the  FOB,  and  the  object’s  speed  is  the  slowest,  then  the  resulting  data  fell 
outside  the  lower  confidence  bound.  Since  the  QRF  does  not  perform  an  interdiction 
unless  either  a  report  is  issued  by  the  UAV  or  the  threat  is  within  the  surveillance  region 
of  the  QRF,  then  for  the  six  design  points,  further  analysis  must  be  performed  to  see  if  the 
results  presented  in  Figure  19  are  due  to  variability  in  the  model. 

4.  Conclusion 

The  prediction  models  presented  based  on  a  cross-validation  technique  provided 
adequate  models  for  each  of  the  response  variables.  Further  analysis  must  be  performed 
for  percentage  of  threats  (Figure  17)  interdicted  based  on  the  object’s  velocity.  Through 
follow-on  analysis,  an  employment  strategy  that  incorporates  perturbing  the  object’s 
velocity  may  provide  an  increased  efficiency  in  predicting  the  interdiction  of  threats. 
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The  model  that  was  developed  for  the  time  to  aequire  threats  (Figure  19) 
adequately  fit  the  subset  of  additional  simulation  experiments.  Through  the  analysis  that 
explores  the  impaet  of  object  velocity,  better  prediction  models  may  be  developed. 
Through  this  analysis,  insights  regarding  each  significant  factor  and  relative  influence 
may  be  explored. 

The  prediction  model  (Figure  20)  provides  insights  with  respect  to  the  interaction 
between  object  velocity  and  the  QRF’s  patrol  employment  strategy.  The  prediction 
interval  used  an  alpha  level  of  0.05,  which  means  that  there  is  a  5%  chance  of  committing 
a  Type  I  error.  Since  a  95%  prediction  interval  is  created,  then  the  prediction  interval 
should  encapsulate  95%  of  the  data.  Further  analysis  should  either  conclude  that  the  9% 
of  the  design  points  that  were  outside  of  the  lower  prediction  band  for  mean  distance  to 
FOB  was  either  due  to  noise,  or  that  this  influence  is  significant  enough  to  impact  the 
simulation.  Based  on  current  analysis,  this  differentiation  cannot  be  concluded  and  is 
resolved  with  additional  analysis. 

C.  EXPLORATION  OF  OBJECT’S  VELOCITY  WITH  RESPECT  TO  EACH 

RESPONSE 

Due  to  a  small  percentage  of  data  points  falling  outside  of  the  prediction  intervals, 
further  analysis  was  performed  to  glean  insight  into  the  behavior  of  the  simulation.  The 
prediction  models  were  created  based  on  fixing  the  object’s  velocity  at  one,  two  and  three 
and  performing  analysis  on  each  of  these  data  sets.  A  linear  regression  model  was  created 
and  a  prediction  model  was  obtained.  A  subset  of  each  of  the  object’s  velocity  design 
points  was  reserved  to  fit  with  the  prediction  models.  The  analysis  and  insights  obtained 
from  utilizing  this  technique  follow. 

1.  Object  Velocity  Fixed  at  the  Maximum  Speed 

The  figures  below  represent  each  of  the  prediction  models  with  their  respective 
responses  overlaid.  Figure  22  shows  that  all  of  the  data  points  for  the  object’s  velocity 
fixed  at  the  maximum  velocity  fall  within  the  95%  prediction  interval  derived.  Some 
useful  insights  can  be  obtained  from  these  pictures  by  observing  that  there  appears  to  be 
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two  distinct  data  intervals  in  Figure  22A.  This  is  due  to  the  patrolling  oharaeteristies  of 
the  QRF,  the  lower  data  set  being  the  QRF  is  patrolling  and  the  upper  set  of  data  is  the 
QRF;  remaining  at  the  FOB.  Therefore,  from  this  insight,  if  the  AOI  eannot  be  altered  to 
reduce  the  speed  of  the  objeet,  then  the  QRF  should  remain  at  the  FOB.  Another  factor 
that  influences  the  results  is  the  inter-arrival  time  of  the  objeets.  By  choosing  an  AOI  that 
minimizes  the  traffie  flow  or  by  establishing  checkpoints  to  minimize  the  ineoming 
objeets,  then  the  Blue  Force  ean  aequire  more  threats. 

Figure  22C  also  shows  two  distinet  data  intervals,  whieh  is  due  to  whether  the 
QRF  patrols  or  not.  If  the  QRF’s  employment  technique  is  to  patrol,  then  the  standoff 
distanee  from  the  FOB  is  inereased;  eonversely,  if  the  QRF  remains  at  the  FOB,  then  the 
standoff  distance  is  deereased.  Therefore,  the  QRF  patrolling  affeets  both  the  pereentage 
of  threats  interdicted,  as  well  as  the  mean  distance  to  FOB;  however,  the  benefits  of  one 
is  detrimental  to  the  other. 


(A)  (B)  (C) 

Figure  22;  95%  predietion  model  fit  for  the  (A)  pereentage  of  threats  interdieted,  (B) 

time  to  acquire  threats,  (C)  mean  distance  from  FOB  with  an  overlay  plot  of  the  data 
obtained  from  a  subset  of  the  fixed  maximum  speed  design  points 
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Figure  23:  Prediction  profiler  for  (A)  Log  Odds  transformation  for  percentage  of 

threats  interdicted,  (B)  time  to  acquire  threats,  (C)  mean  distance  from  FOB 


Figure  23  illustrates  the  prediction  profiler  for  each  response  variable.  Figure  23A 
shows  that  the  dominant  factor  for  the  percentage  of  threats  interdicted  is  whether  the 
QRF  patrols  (1)  or  not  (-1).  For  both  the  time  to  acquire  (Figure  23B)  threats  and  mean 
distance  from  FOB  (Figure  23 C),  the  dominant  factors  are  object  velocity  and  QRF 
patrol.  Based  on  this  analysis,  techniques  to  reduce  the  object’s  velocity  increases  the 
time  to  acquire  the  threats  as  well  as  increase  the  standoff  distance  from  the  FOB.  Also, 
Figure  23C  illustrates  that  if  the  QRF  patrols,  then  the  standoff  distance  is  maximized. 

Analysis  of  the  model  with  the  objects  at  their  maximum  speed,  the  dominating 
factor  is  the  object’s  velocity.  To  ensure  that  a  large  number  of  threats  are  interdicted,  the 
QRF  should  remain  stationary  at  the  FOB.  However,  if  the  commander  wants  to 
maximize  the  standoff  distance  to  the  FOB,  then  a  patrolling  QRF  should  be  employed. 
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Through  this  analysis,  a  trade-off  exists  between  interdieting  threats  or  maximizing  the 
standoff  distanee  and  an  understanding  of  the  impaet  of  ehoosing  one  over  the  other  must 
be  made  elear. 

2.  Object  Velocity  Fixed  at  the  Slowest  Speed 

In  this  study,  the  SASIO  model  simulated  the  design  matrix  fixing  the  objeet’s 
veloeity  to  the  minimum  value.  Taking  a  subset  of  this  output  with  the  eross-validation 
design  matrix,  teehniques  were  used  to  ereate  a  linear-regression  model.  Figure  24,  shows 
that  all  but  one  data  point  fall  within  the  predietion  model  and  one  ean  eonelude  that  the 
predietion  models  are  adequate. 

One  notes  that  there  are  two  distinet  oases  represented  in  Figure  24 A,  as  with  the 
objeot’s  veloeity  set  to  the  maximum  speed.  By  using  the  predietion  profiler  (Figure  25 A) 
in  oonjunotion  with  the  predietion  plot,  the  most  signifioant  faotor  is  whether  the  QRF 
patrols  or  not.  The  insights  obtained  from  this  analysis  are  that  the  QRF  should  remain 
stationary  at  the  FOB  to  interdiot  most  of  the  threats.  Like  the  analysis  performed  for  the 
maximum  objeot  speed,  the  AOI  should  be  ohosen  or  oheokpoints  should  be  established 
to  minimize  the  traffio  flow  to  the  FOB. 

The  95%  predietion  model  (Figure  24B)  adequately  prediets  the  data  ereated  by 
minimizing  the  objeet  veloeity.  The  predietion  profiler  (Figure  25B)  demonstrates  that 
the  most  signifioant  factor  remains  object  velocity.  This  follows  that  if  an  object  is  slow, 
then  the  time  to  acquire  this  threat  increases.  This  is  due  to  the  increased  time  between 
the  object  entering  into  the  AOI  and  an  interdiction  report  being  made. 

From  the  analysis  performed  with  respect  to  the  mean  distance  from  FOB,  the 
prediction  model  adequately  predicts  the  data.  Two  distinct  cases  arise  in  this  graph  and 
they  are  due  to  whether  the  QRF  patrols  or  not.  If  the  QRF  remains  stationary  at  the  FOB, 
then  the  mean  distance  from  the  FOB  is  decreased  as  shown  in  Figure  25C.  However,  if 
the  QRF  patrols,  then  the  standoff  distance  is  maximized. 
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▼  E  Overlay  Plot 
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Figure  24:  95%  prediction  model  fit  for  the  (A)  percentage  of  threats  interdicted,  (B) 

time  to  acquire  threats,  (C)  mean  distance  from  FOB  with  an  overlay  plot  of  the  data 
obtained  from  a  subset  of  the  fixed  maximum  speed  design  points 
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Figure  25:  Prediction  profder  for  (A)  Log  Odds  transformation  for  percentage  of 

threats  interdicted,  (B)  time  to  acquire  threats,  (C)  mean  distance  from  FOB 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A,  CONCLUSIONS 

The  factors  being  used  for  this  analysis  should  be  categorized  in  two  groups: 
factors  that  are  controllable  (Blue  Force  employment  techniques)  and  those  that  are 
uncontrollable  (enemy  tactics).  Once  the  factors  are  broken  down  into  these  groups,  the 
Blue  Force  employment  techniques  can  be  better  explored  to  maximize  the  percentage  of 
threats  interdicted,  minimize  time  to  interdict  these  threats,  and  maximize  the  distance 
from  the  FOB  the  threats  are  interdicted,  as  well  as  seek  methods  to  best  disrupt  the 
enemy’s  tactics. 

1,  Blue  Force  Employment 

When  using  the  UAV,  having  sensors  with  minimal  errors  results  in  an  increased 
identification  rate.  This  factor  has  a  minor  effect  on  influencing  the  percentage  of  threats 
interdicted.  This  is  due  to  the  ability  of  the  UAV  to  provide  an  increased  identification 
range  (i.e.,  increased  standoff  identification)  and  can  aid  the  commander  in  properly 
responding  to  a  threat  prior  to  its  arrival  at  the  FOB. 

The  next  employment  technique  is  to  decide  whether  the  QRF  should  patrol  or 
not.  Based  on  the  analysis,  the  percentage  of  threats  interdicted  decreases  if  the  QRF 
patrols;  however,  the  time  till  interdiction  decreases  with  patrols  and  the  mean  distance  to 
FOB  is  maximized  as  the  QRF  patrols.  The  former  effect  is  potentially  due  to  the  QRF 
searching  a  different  road  while  performing  its  patrol  and  not  being  able  to  perform 
surveillance  on  the  other  road.  There  is  a  larger  decrease  in  the  percentage  of  threats 
interdicted  over  the  benefits  gained  in  time  to  interdict  or  mean  distance  to  FOB.  Based 
on  having  only  a  single  QRF  to  patrol,  one  would  recommend  maintaining  the  QRF 
stationary  until  either  a  report  is  issued  by  the  UAV  or  observed  by  the  QRF  while  in 
close  proximity  to  the  FOB. 
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Based  on  the  insights  provide  by  the  model,  the  Blue  Foree  ean  sueeessfully 
perform  its  operation  utilizing  a  single  QRF  and  UAV.  More  analysis  must  be  performed 
on  the  effeet  of  an  inerease  in  number  of  UAVs  and  QRF  units  employed.  However, 
preliminary  eonsideration  of  utilizing  multiple  UAVs  lead  to  reeommendations  sueh  as  to 
employ  the  UAVs  at  maximal  distance  to  ensure  rapid  identification.  If  there  are  two 
QRFs,  then  based  on  the  analysis,  the  commander  should  allow  one  QRF  to  patrol  the 
area,  while  maintaining  one  QRF  at  the  FOB.  This  allows  for  FOB  protection  with  the 
stationary  QRF  while  an  increased  interdiction  range  with  the  patrolling  QRF. 

2,  Blue  Force  Actions  External  to  Current  Assets 

The  predominate  threat  factor  is  the  speed  at  which  the  object  travels  down  a 
road.  As  the  speed  is  increased,  then  all  three  responses  are  degraded.  Therefore,  based 
on  this  analysis,  the  Blue  Force  can  set  up  blockades  or  other  obstacles  in  order  to  reduce 
the  speed  at  which  all  objects  traverse  down  the  road. 

The  other  factor  that  the  Blue  Force  can  influence  is  the  road  selection  of  the 
objects.  If  each  road  is  limited  to  travelling  down  a  single  road,  then  each  response 
increases  in  the  Blue  favor.  Setting  up  checkpoints  prior  to  the  entrance  into  the  AOI  can 
counteract  this  problem.  These  checkpoints  can  be  provided  by  the  indigenous  population 
that  can  provide  intelligence  to  the  Blue  Force,  but  whose  primary  goal  is  to  perturb  the 
flow  of  traffic  down  each  road. 

B,  ONGOING  RESEARCH 

Ongoing  research,  which  utilize  SASIO  model  framework,  studies  the  potential 
improvement  in  effectiveness  by  increasing  the  available  surveillance  and  interdiction 
assets.  Another  study  utilizing  the  SASIO  model  framework  is  a  game -theoretic  approach 
that  encompasses  an  adversarial  threat  and  potential  strategies  to  combat  these  threats. 
The  study  explores  current  Blue  Force  strategies  and  compares  those  strategies  with 
possible  improvements  based  on  an  effective  use  of  surveillance  and  interdiction  assets. 

The  current  SASIO  model  implementation  requires  either  utilizing  command-line 
arguments  or  a  graphical  user  interface  (GUI)  to  run  the  program.  Current  work  is  to 
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continue  to  evolve  the  GUI  interface  as  new  concept  and/or  ideas  arise  such  as:  switching 
from  a  screening  design  to  sensitivity  analysis,  implementing  the  desired  number  of 
replications,  and  choosing  the  scenario  based  on  user  inputs.  As  seen  in  Figure  26,  the 
GUI  is  currently  used  for  the  SASIO  scenario  posed  in  this  thesis;  however,  the  evolution 
of  this  program  allows  an  easy  to  command  shell  to  execute  tactical  scenarios. 


^  ^  ^ _ SASIOMnsight  —  Tactical  Installation  Protection 
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